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MULTI-TASKING  ASSESSMENT  FOR  PERSONNEL  SELECTION  AND 
DEVELOPMENT 


EXECUTIVE  SUMMARY 


Research  Requirement: 

The  requirement  to  perform  more  than  one  task  within  a  limited  period  of  time  is  prevalent 
in  work  environments,  and  .the  number  of  jobs  that  require  multi-tasking  (MT)  may  be  rising. 
Unfortunately,  research  has  shown  that  MT  has  serious  negative  consequences  to  morale  and 
performance.  Yet,  research  has  also  shown  that  some  individuals  seem  resistant  to  the  negative 
effects  of  MT,  and  even  seem  to  thrive  on  the  challenge.  If  individuals  vary  in  their  ability  to 
multi-task,  it  should  be  possible  to  develop  a  screening  test  that  could  be  used  for  selection  or 
placement  purposes.  A  reliable  and  valid  assessment  instrument  would  be  highly  useful  to  many 
different  civilian  and  military  organizations.  Training  costs,  turnover,  and  attrition  could  be 
reduced  for  many  MT  jobs  by  using  the  test  to  select  those  individuals  who  would  perform  well 
on  the  job. 

However,  the  indubitable  variation  among  MT  environments  calls  into  question  the  idea 
that  a  single  measure  of  MT  ability  is  sufficient  to  the  task  of  predicting  performance  in  a  wide 
range  of  MT  jobs.  It  is  unlikely  that  any  single  measure  of  MT  ability  will  equally  predict 
performance  in  varying  MT  environments.  Therefore,  it  may  be  necessary  to  create  multiple 
measures  of  MT  ability,  each  specific  to  the  unique  job  demands  placed  on  workers  by  different 
MT  environments.  To  do  so,  a  better  understanding  is  needed  of  the  variation  among  MT 
environments,  and  how  that  variation  is  related  to  job  demands  and  individual  abilities. 

A  central  purpose  of  the  present  research  was  to  investigate  variation  that  exists  among 
MT  environments  to  form  a  better  understanding  of  the  job  demands  placed  on  workers  in  these 
environments.  The  primary  technical  objective  of  the  present  research  was  to  design  a 
measurement  approach  that  could  be  used  to  predict  performance  in  different  kinds  of  MT 
environments  -  especially  those  likely  to  be  encountered  by  first-term  Army  enlistees. 


Procedure: 

The  research  was  initiated  with  an  analysis  of  the  psychological  literature  related  to  MT. 
The  purpose  of  the  review  was  to  identify  individual  difference  and  environmental  variables  that 
research  has  shown  affect  performance  in  MT  environments.  The  second  task  was  to  develop  a 
model  of  real-world  MT  environments  that  defines  them,  distinguishes  them  from  non-MT 
environments,  explains  their  commonalities,  and  describes  how  they  vary.  To  accomplish  this 
task,  a  set  of  ten  first-term  Military  Occupational  Specialties  (MOSs)  was  identified  for  study. 
The  jobs  were  then  analyzed  by  interviewing  individuals  who  had  extensive  knowledge  and 
experience  with  each  MOS.  The  interview  results  were  then  used  to  develop  an  MT  environment 
model.  Key  individual  difference  variables  were  matched  to  specific  characteristics  of  MT 
environments  identified  in  the  interviews.  This  mapping  was  then  compared  to  the  assessment 
capabilities  of  an  existing  measure  of  MT  ability.  Additional  features  were  then  identified  that 
are  absent  from  the  existing  measure,  but  could  be  incorporated  into  new  versions.  A 


capabilities  of  an  existing  measure  of  MT  ability.  Additional  features  were  then  identified  that 
are  absent  from  the  existing  measure,  but  could  be  incorporated  into  new  versions.  A 
measurement  approach  was  then  designed  based  on  this  analysis.  The  resulting  measurement 
approach  addresses  both  environmental  and  individual  assessment  needs.  A  plan  was  developed 
for  constructing  the  measurement  instruments,  and  conducting  validity  studies. 


Findings: 

Based  on  the  literature  review,  a  list  was  compiled  of  dimensions  upon  which  MT 
environments  likely  vary.  The  compilation  focused  on  the  variables  that  other  basic  and  applied 
researchers  have  investigated,  that  seem  the  most  important  characteristics  of  MT  environments, 
and  that  seem  the  most  relevant  to  potential  individual  differences. 

The  interview  responses  showed  that  the  MOS’s  investigated  vary  on  many  dimensions. 
Three  different  dimensions  were  most  salient  in  the  data:  the  type  of  multitasking  required,  the 
intensity  of  the  MT,  and  the  consequences  of  failure.  Patterns  of  variation  also  emerged  from  the 
data.  Certain  environmental  characteristics  seem  to  cluster  together  in  MT  jobs,  such  as  high 
levels  of  autonomy  in  prioritizing  tasks  and  the  presentation  of  ill-specified  problem  spaces. 
Based  on  these  findings,  a  model  was  developed  of  MT  environments.  The  model  proposes  that 
the  variance  among  MT  environments  is  multidimensional.  Some  dimensions  on  which  MT 
environments  vary  have  the  result  of  placing  different  demands  on  workers,  while  other 
dimensions  do  not.  The  model  states  that  key  dimensions  cluster  to  form  different  types  or  kinds 
of  environments.  Therefore,  it  presents  a  typology  of  MT  environments  that  captures  many 
important  differences,  particularly  differences  that  place  different  kinds  of  demands  on  workers. 
The  typology  includes  three  kinds  of  environments.  The  first  kind  requires  high  levels  of 
decision-making  capability.  The  second  kind  presents  workers  with  well-defined  problems  that 
require  routine  decision-making.  A  third  kind  is  characterized  by  its  multiple  sources  of 
information.  The  three  types  of  environments  appeared  to  place  differing  emphasis  on  the  ability 
to  prioritize,  make  decisions,  and  manage  time,  among  other  key  job  demands. 

The  model  was  used  to  guide  the  design  of  a  measurement  approach,  which  could  be  used 
to  evaluate  multitasking  ability  in  a  wide  variety  of  environments.  The  measurement  approach  is 
actually  a  relatively  complex  assessment  system,  the  Multi-Tasking  Assessment  System  (MTAS). 
The  MTAS  is  designed  to  assess  individuals’  MT  abilities,  as  well  as  work  environments  to 
determine  (1)  whether  an  environment  qualifies  as  an  MT  work  setting  and  (2)  the  kinds  of  MT 
demands  it  places  on  workers.  The  MTAS  consists  of  two  main  components:  the  Environment 
Assessment  Tool  (ENVAT),  which  assesses  the  MT  requirements  of  work  environments,  and  a 
test  component  that  includes  three  basic  versions  of  a  Multi-Tasking  Ability  Test  (MTAT),  each 
of  which  assesses  individuals’  performance  in  a  particular  type  of  MT  environment.  The  testing 
component  also  includes  three  sub-versions  of  each  MTAT  test,  which  are  adaptively 
administered  and  are  designed  to  tap  performance  differences  associated  with  environmental 
variation  at  three  levels  of  intensity. 

Utilization  and  Dissemination  of  Findings: 

The  model  developed  in  the  present  research  advances  knowledge  about  how  different 
environments  place  different  demands  on  workers.  It  offers  the  opportunity  to  further  advance 


complex  real-world  environments,  and  away  from  the  current  emphasis  on  the  simpler  laboratory 
setting. 

The  MTAS  is  a  flexible  system  that  provides  tailored  assessments  of  particular  MT 
environments  as  well  as  individual  tests  that  tap  the  demands  required  by  those  environments. 
The  flexibility  of  the  proposed  MTAS  allows  it  to  be  used  both  as  a  selection  tool  and  as  a 
placement  tool  for  prospective  job  candidates.  Likewise,  a  job  placement  counselor  might 
administer  all  three  versions  of  the  MTAT  to  a  job  seeker,  then  use  information  provided  in  the 
test  manual  to  provide  guidance  about  which  workplace  environments  the  job  seeker  would  be 
best  suited  for. 
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MULTI-TASKING  ASSESSMENT  FOR  PERSONNEL  SELECTION  AND 
DEVELOPMENT 


INTRODUCTION _ 

The  requirement  to  perform  more  than  one  task  within  a  limited  period  of  time  is  prevalent 
in  work  environments,  and  the  number  of  jobs  that  require  multi-tasking  (MT)  may  be  rising.  In 
recent  years,  many  military  and  civilian  organizations  have  reduced  staff  in  an  effort  to  boost 
productivity  and  decrease  costs.  In  most  cases,  task  load  was  not  concomitantly  reduced.  Hence, 
the  effect  of  staff  reductions  has  been  to  increase  MT  requirements.  While  it  is  possible  that 
productivity  was  enhanced  by  these  actions,  MT  has  a  significant  downside.  Research  has  shown 
that  MT  has  serious  negative  consequences  to  morale  and  performance  (Bogner,  1994;  Cook  & 
Woods,  1994;  Proctor,  Wang,  and  Pick,  1998;  Wickens,  1980;  Kahneman,  1973;  Meyer  & 
Kieras,  1997).  Field  studies  have  associated  MT  with  increased  error,  bumout,  stress,  and 
attrition  in  work  environments  such  as  air  traffic  control  (ATC),  nursing,  and  emergency 
dispatching  (Franklin  &  Hunt,  1993;  Joslyn  &  Hunt,  1998).  Moreover,  the  ostensible 
productivity  benefits  of  MT  may  be  illusory  because  overall  performance  is  typically  slowed 
under  MT  conditions  (Meyer  &  Kieras,  1997). 

Yet,  research  has  also  shown  that  some  individuals  seem  resistant  to  the  negative  effects  of 
MT,  and  even  seem  to  thrive  on  the  challenge.  Laboratory  studies  have  shown  that  some  people 
do  not  show  the  typical  performance  decrement  associated  with  multi-tasking  conditions 
(Rubinstein,  Meyer  &  Evans,  2001;  Schumacher,  Seymour,  Glass,  Fencsik,  Lauber,  Kieras, 
Meyer,  2001).  Other  studies  have  shown  that  individual  differences  in  cognitive  processing  and 
personality  are  related  to  performance  differences  observed  under  MT  conditions  (e.g., 
Brookings,  &  Damos,  1991;  Dickman,  1990;  Dickman  &  Meyer,  1988;  Ishizaka,  Marshall, 
Conte,  2001).  For  example,  some  individuals  seem  to  be  better  able  to  prioritize  multiple  tasks 
and  switch  mental  set  with  each  new  task  (e.g.,  Burgess,  2000;  Fischer,  Morrin,  &  Joslyn,  2003; 
Gopher,  1982;  Sohn  and  Anderson,  2001;  Wickens,  1999).  Extraversion  and  Type-A  behavior 
pattern  (TABP)  have  also  been  shown  to  influence  performance  in  laboratory  MT  conditions 
(Conte,  Rizzuto,  &  Steiner,  1999;  Lieberman  and  Rosenthal,  2001). 

Although  a  variety  of  cognitive  and  personality  individual  difference  variables  have  been 
shown  to  influence  performance  in  MT,  their  relative  importance  probably  depends  on  the 
particular  environment  being  considered.  It  is  likely  that  some  variables,  such  as  the  ability  to 
effectively  prioritize,  are  much  more  critical  to  successful  performance  in  some  environments 
than  in  others.  This  is  simply  because  MT  environments  undoubtedly  vary  in  the  kind  of 
demands  they  make  on  workers’  abilities.  A  better  understanding  of  the  relationships  among 
individual  variables  and  environmental  variables  is  needed  to  fully  understand  why  some  people 
perform  better  than  others  in  MT  settings.  In  summary,  it  appears  that  individual  differences  in 
elemental  cognitive  abilities  and  personality  play  an  important  role  in  determining  MT 
performance.  However,  a  comprehensive  model  is  needed  of  individual  difference  variation  as  it 
relates  to  environmental  variation. 

If  individuals  vary  in  their  ability  to  multi-task,  it  should  be  possible  to  measure  that  ability 
and  use  the  measurement  to  predict  future  performance  in  MT  environments.  In  other  words,  it 
should  be  possible  to  develop  a  screening  test  of  MT  ability  that  could  be  used  for  selection  or 
placement  purposes.  A  reliable  and  valid  assessment  instrument  would  be  highly  useful.  Many 
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different  civilian  and  military  organizations  could  use  it  to  identify  individuals  who  are  likely  to 
perform  well  in  MT  jobs,  and  those  that  will  probably  perform  poorly.  Training  costs,  turnover, 
and  attrition  for  many  MT  jobs  could  be  reduced  by  using  the  test  to  select  those  individuals  who 
would  perform  well  on  the  job. 

Indeed,  several  potential  measures  of  MT  ability  have  been  developed  (e.g.,  Burgess, 
Veitch,  de  Lacy  Costello,  &  Shallice,  2000;  Comstock  &  Amegard,  1992;  Joslyn  &  Hunt,  1998; 
Proctor,  Wang,  and  Pick,  1998;  Shallice  &  Burgess,  1991;  Yee,  Hunt,  &  Pellegrino,  1991)  and 
research  is  underway  to  develop  a  new  test  based  on  Joslyn’s  and  Hunt’s  (1998)  original  work 
(Fischer,  Morrin,  &  Joslyn,  2003).  Some  of  these  measures  have  been  developed  for 
neurologically  disabled  populations  (Burgess,  Veitch,  de  Lacy  Costello,  &  Shallice,  2000; 
Shallice  &  Burgess,  1991),  making  them  unsuitable  for  selection  and  placement  purposes  for  use 
with  normal  populations.  The  predictive  validity  of  several  other  instruments  has  not  been 
evaluated  or  has  failed  to  meet  the  criteria  necessary  for  a  selection  or  placement  test  (Comstock 
&  Amegard,  1992;  Proctor,  Wang,  and  Pick,  1998;  Yee,  Hunt,  &  Pellegrino,  1991). 

The  most  promising  existing  instrument  that  purports  to  measure  individual  differences  in 
MT  ability  is  the  Abstract  Decision  Making  task  (ADM),  originally  developed  by  Joslyn  and 
Hunt  (1998).  It  stands  heads  above  other  potential  assessment  instruments  because  it  has  been 
shown  to  predict  performance  in  two  very  different  MT  work  environments  (dispatching  and  air 
traffic  control)  at  unusually  high  levels  of  accuracy,  accounting  for  nearly  50%  of  the  variance  in 
simulated  dispatching  performance  and  over  25%  of  the  variance  in  simulated  ATC.  It  is  a 
computer-based  task  that  can  be  simply  and  quickly  administered  with  minimal  training,  and 
requires  no  domain  knowledge.  For  these  reasons,  effort  is  underway  to  develop  the  ADM 
laboratory  task  as  a  predictive  measure  of  MT  ability.1 

The  new  test  that  is  based  on  ADM  and  is  currently  under  development  (Multi-Tasking 
Ability  Test  (MTAT);  Fischer,  Morrin,  &  Joslyn,  2003)  is  designed  to  measure  the  ability  to 
perform  cognitive  tasks  that  are  common  to  most  MT  environments.  Cognitive  task  analysis 
(Fischer,  Morrin,  &  Joslyn,  2003)  of  the  MTAT  suggests  that  it  requires  test  takers  to  employ 
cognitive  abilities  that  are  demanded  by  many,  if  not  all,  MT  environments.  For  example,  the 
MTAT  requires  the  test  taker  to  employ  his/her  prospective  memory.  Individuals  who  do  not 
effectively  use  their  prospective  memory  do  poorly  on  the  MTAT,  and  also  perform  poorly  in 
MT  environments.  Similarly,  the  MTAT  places  heavy  demands  on  working  memory,  which  is  a 
cognitive  resource  individuals  must  utilize  effectively  to  perform  well  in  most  MT  environments. 

Although  research  has  shown  that  the  ADM  task  predicts  performance  in  two  MT 
environments  at  unusually  high  accuracy  levels,  it  also  shows  that  the  ADM  task  does  a  better 
job  of  predicting  skill  in  emergency  dispatching  than  in  ATC  (Joslyn  &  Hunt,  1998).  Because  the 
new  MTAT  is  closely  based  on  ADM,  it  is  likely  that  the  MTAT  will  also  vary  in  its  ability  to 
predict  performance  in  different  MT  environments.  Differential  predictive  validity  is  expected 
because  the  environments  themselves  vary  along  numerous  dimensions.  Fischer,  Morrin,  & 
Joslyn  (2003)  argue  that  all  MT  environments  share  several  characteristics  that  the  MTAT  also 
shares.  However,  substantial  variation  also  exists  among  MT  environments.  Some  dimensions  on 
which  MT  environments  vary  are  not  simulated  by  the  MTAT.  Other  variables  are  represented 


1  See  Fischer,  Morrin,  &  Joslyn  (2003)  for  a  thorough  discussion  of  existing  measures  of  MT 
ability,  and  the  relative  advantages  of  the  ADM  task  as  a  potential  assessment  instrument. 
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by  the  MTAT,  but  only  one  end  of  the  dimension  is  simulated.  For  example,  some  jobs  offer  the 
opportunity  to  create  novel  and  creative  solutions  to  ill-defined  problem  spaces.  In  contrast, 
others  present  problem  spaces  for  which  only  a  single  solution  is  appropriate.  Effective  decision 
making  in  environments  that  present  well-defined  problems  depends  on  the  ability  to  accurately 
assess  the  situation  and  match  it  to  the  single  appropriate  solution,  not  to  create  a  novel  solution. 
The  problems  offered  by  the  MTAT  do  not  require  creative  solutions;  there  is  only  one  correct 
solution.  It  is,  therefore,  likely  that  the  MTAT  would  be  a  better  predictor  of  performance  in 
environments  that  present  well-defined  problem  spaces  to  the  worker. 

The  indubitable  variation  among  MT  environments  calls  into  question  the  idea  that  a  single 
measure  of  MT  ability  is  sufficient  to  the  task  of  predicting  performance  in  a  wide  range  of  MT 
jobs.  It  is  unlikely  that  the  MTAT  will  equally  predict  performance  in  MT  environments  that 
vary  considerably  in  the  types  of  job  demands  they  place  on  workers.  The  new  MTAT  will 
probably  be  a  less  powerful  predictor  of  performance  for  some  MT  environments  than  for  others. 
Therefore,  it  may  be  necessary  to  create  multiple  measures  of  MT  ability,  perhaps  multiple 
versions  of  the  MTAT,  each  specific  to  the  unique  job  demands  placed  on  workers  by  different 
MT  environments.  An  idealistic  solution  would  be  to  create  and  tailor  separate  MT  ability 
measures  for  each  MT  environment  for  which  a  predictor  was  desired.  Yet,  it  is  not  practical  to 
create  a  tailored  MT  ability  measure  for  every  MT  job  that  exists.  It  seems  a  tradeoff  must  be 
made  between  practical  realities  and  optimal  predictive  validity. 

Perhaps  the  solution  to  the  tension  between  practical  concerns  and  the  need  for  the  most 
accurate  predictors  is  a  better  understanding  of  the  relationships  among  individual  difference  and 
environmental  variables.  It  is  possible  that  environments  vary  in  typical  and  systematic  ways, 
and  that  this  systematic  variation  is  related  to  individual  difference  variables.  It  might  be  possible 
to  identify  environmental  types,  which  would  be  characterized  by  the  values  they  took  along  key 
variables  in  a  multi-dimensional  space.  In  other  words,  there  may  be  major  differences  among 
MT  environments  that  can  be  dimensionalized  so  as  to  identify  different  kinds  or  clusters  of  MT 
jobs.  It  is  further  possible  that  these  different  kinds  of  jobs  could  be  differentiated  by  the 
different  demands  they  place  on  workers  and,  hence,  the  different  kinds  of  individual  capabilities 
they  require.  If  environmental  types  could  reliably  be  identified,  and  the  variation  among 
environments  is  related  to  individual  difference  abilities,  then  it  would  also  be  possible  to 
develop  measures  of  the  abilities  demanded  by  the  environmental  types.  Fewer  assessment 
instruments  would  be  necessary  to  accurately  predict  MT  performance  if  a  useful  typology  of 
MT  environments  could  be  developed. 

A  central  purpose  of  the  present  research  was  to  investigate  variation  that  exists  among 
MT  environments  to  form  a  better  understanding  of  the  job  demands  placed  on  workers  in  these 
environments.  Researchers  understand  that  MT  environments  vary  greatly  in  superficial 
characteristics  and  in  the  background  knowledge  they  require.  What  scientists  do  not  yet 
understand  is  whether  MT  environments  vary  in  ways  that  are  related  to  variation  in  individuals’ 
cognitive  abilities  or  personality.  A  literature  review  was  conducted  to  identify  key  individual 
difference  variables  and  key  environmental  variables  shown  to  affect  performance  in  MT 
settings.  To  further  investigate  variation  among  MT  environments,  a  series  of  interviews  was 
conducted  with  subject  matter  experts  (SME)  in  ten  different  military  jobs.  The  ten  environments 
were  selected  because  they  appeared  to  vary  in  the  degree  and  kind  of  MT  capabilities  they 
demanded.  Based  on  data  gathered  in  the  interviews,  the  results  of  the  literature  review,  and  our 
previous  research,  a  comprehensive  model  was  developed  that  explains  the  relationship  between 
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environmental  variation  among  MT  work  settings  to  individual  difference  abilities.  The  model 
was  then  used  to  guide  design  of  a  measurement  approach  for  assessing  MT  ability  variation 
pertinent  to  different  kinds  of  MT  environments.  The  remainder  of  this  report  discusses  the 
methods  and  findings  of  the  present  research. 

OVERVIEW  OF  PHASE  I  RESEARCH 


The  primary  technical  objective  of  the  present  research  was  to  develop  a  plan  for  the 
design  of  a  set  of  reliable,  valid,  and  practical  measures  of  multi-tasking  ability  that  could  predict 
performance  in  different  kinds  of  MT  environments  -  especially  those  likely  to  be  encountered 
by  first-term  Army  enlistees.  Supporting  technical  objectives  were  to  (1)  develop  a  model  of  MT 
environments  that  explains  their  commonalities  and  differences  and  (2)  identify  the  individual 
difference  variables  that  differentially  influence  performance  in  different  MT  environments.  The 
utility  of  multiple  measures,  which  the  present  research  identified  as  needed,  will  be  to  select 
applicants  that  are  most  likely  to  succeed  in  a  variety  of  MT  jobs,  thereby  reducing  training  costs 
and  attrition. 

Figure  1  depicts  the  strategy  employed  in  the  present  research.  The  research  was  initiated 
by  first  conducting  an  analysis  of  psychological  literature  related  to  MT  in  order  to  identify 
individual  difference  and  environmental  variables  that  research  has  shown  contribute  to 
performance  in  MT  environments,  whether  in  the  laboratory  or  in  the  field.  The  primary  product 
of  Task  1  was  a  comprehensive  and  detailed  set  of  cognitive  and  non-cognitive  individual 
variables  as  well  as  environmental  variables  that  are  related  to  successful  MT  performance. 


Figure  1.  Project  tasks  and  deliverables. 
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The  second  task  was  to  develop  a  model  of  MT  environments  that  explains  how  they  vary 
and  what  they  have  in  common.  To  accomplish  this  task,  we  first  developed  criteria  to  guide  the 
selection  of  a  set  of  first-term  MOSs  to  study.  To  maximize  our  ability  to  distinguish 
environments  that  vary  in  degree  or  kind,  we  selected  10  first-term  MOSs.  We  then  analyzed  the 
jobs  by  interviewing  individuals  who  had  extensive  knowledge  and  experience  with  each  MOS. 
The  interview  data  was  used  to  develop  abstract  profiles  of  different  types  of  MT  environments. 
The  resulting  model  of  MT  environments  delineates  the  factors  that  distinguish  one  MT 
environment  from  another,  as  well  as  identifies  the  variables  that  distinguish  these  jobs  from 
non-MT  environments. 

Based  on  the  products  of  Tasks  1  and  2,  a  measurement  approach  was  designed  (Task  3) 
by  first  matching  key  individual  variables  to  specific  characteristics  of  multi-tasking 
environments.  This  mapping  was  then  compared  to  the  assessment  capabilities  of  the  MTAT. 
Having  identified  individual  variables  that  are,  and  are  not,  measured  by  the  MTAT,  additional 
features  were  then  identified  that  may  be  incorporated  into  new  versions  of  the  MTAT.  A 
measurement  approach  was  then  developed  that  addresses  both  environmental  and  individual 
assessment  needs. 

In  Task  4  a  plan  was  developed  for  constructing  the  measurement  instruments,  and 
conducting  validity  studies.  The  findings  and  products  of  the  research  effort  are  described  in 
greater  detail  in  the  remainder  of  this  report. 

REVIEW  OF  LITERATURE _ 

To  identify  individual  difference  factors  related  to  performance  in  MT  environments,  an 
extensive  review  of  the  literature  relevant  to  MT  was  conducted.  A  variety  of  sources  was 
reviewed  including  published  laboratory  experiments,  theoretical  articles,  and  applied  research. 
Literature  from  a  variety  of  academic  and  government  databases  was  queried,  such  as  ERIC 
(educational  literature),  NTIC  and  DTIC  (military  and  federal  government  literature),  PsycINFO 
(psychological  literature),  and  Infosurf  and  Elsevier  Science  Direct  Electronic  Journals 
Databases.  The  keywords  used  in  the  search  included  terms  such  as:  MT,  multi-tasking,  multi¬ 
task,  time  sharing,  dual  tasks,  task  switching,  time-pressured  decision-making,  and  individual 
differences.  Research  on  some  of  the  key  individual  difference  traits  and  cognitive  factors  that 
contribute  to  MT  performance  was  also  reviewed,  such  as  information  coordination, 
polychronicity,  divided  attention,  selective  attention,  and  prospective  memory.  Additional 
sources  were  identified  from  the  reference  sections  of  reviewed  sources,  as  well  as  from  Internet 
searches  of  leading  researchers’  websites. 

Approximately  237  articles  were  selected  for  review.  Many  of  the  articles  were  only 
tangentially  related  to  multi-tasking  or  did  not  appear  to  relate  to  multi-tasking  as  we  defined  it. 
For  example,  some  articles  examined  performance  on  multiple  sequential  tasks  rather  than  on 
multiple  simultaneous  tasks,  which  is  integral  to  our  definition  of  MT.  The  review  also  focused 
on  articles  that  addressed  individual  differences  in  MT  performance  of  non-clinical  adult 
populations.  In  addition,  previous  review  of  MT  literature  had  focused  primarily  on  cognitive 
variables,  so  special  emphasis  was  placed  on  reviewing  sources  relating  personality  traits  to  MT 
performance.  A  coding  sheet  was  used  to  record  relevant  individual  difference  variables  and  how 
they  were  measured,  as  well  as  characteristics  of  the  MT  environment;  and  the  interaction  -  if 
any  -  between  the  two. 
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Summary  of  Results 

Many  of  the  reviewed  studies  examined  the  correlation  among  individual  differences 
measures  and  performance  in  real-world  or  laboratory  MT  environments.  For  example,  Conte 
and  Jacobs  (2003)  found  correlations  between  supervisory  ratings  of  job  performance  in  MT 
environments  and  personality  dimensions  such  as  extraversion,  conscientiousness, 
polychronicity,  and  neuroticism.  Other  studies  looked  at  the  relationship  between  personality 
traits  and  task  performance  in  more  controlled  MT  environments.  Szymura  and  Necka  (1998) 
and  Lieberman  and  Rosenthal  (2001),  for  example,  demonstrated  the  relationship  between 
extraversion  and  performance  on  MT  laboratory  tasks.  In  these  studies,  introverts  and  extroverts 
performed  individual  tasks  equally  well,  but  when  required  to  perform  the  tasks  nearly 
simultaneously  (i.e.,  to  multi-task),  introverts’  performance  on  the  secondary  task  tended  to 
decrease.  Other  correlational  studies  focused  more  on  cognitive  skills  and  abilities.  In  Chiles, 
Jennings,  and  West’s  study  (1972),  experimenters  administered  a  battery  of  assessments  to  FAA 
trainees,  then  looked  at  which  ones  best  correlated  with  supervisory  ratings  and  later  on-the-job 
performance  ratings.  They  found  that  even  simple,  abstract  assessment  tasks  were  good 
predictors  of  future  performance  in  real-world  environments.  More  recently,  Heil  (1999) 
administered  a  battery  of  cognitive  tests  (such  as  fluid  intelligence,  working  memory,  and 
reasoning  ability)  to  incumbent  air  traffic  controllers  and  found  that  performance  patterns 
indicated  three  main  factors  or  clusters  of  tasks:  dynamic  tasks  that  require  perceptual  speed  and 
fluid  intelligence,  reasoning  tasks  that  require  application  of  prior  knowledge,  and  analytical  and 
procedural  accuracy  tasks  that  require  memory  for  following  rules. 

The  findings  of  correlational  studies,  however,  were  not  always  consistent.  For  example, 
Jemeic  and  Sverko  (1994)  failed  to  find  significant  correlations  between  personality  factors  and 
MT  task  performance,  though  they  did  find  correlations  between  cognitive  abilities  and  task 
performance.  Delbridge  (2001)  administered  a  battery  of  individual  difference  assessments  - 
including  assessments  of  neuroticism,  extraversion,  intelligence,  and  Type  A  Behavior  Pattern  - 
to  college  students,  and  looked  at  how  the  assessments  correlated  with  performance  on 
laboratory  MT  environment  tasks  both  in  terms  of  accuracy  and  persistence.  Contrary  to  results 
found  in  some  of  the  other  studies,  there  were  no  significant  correlations  found  between  MT 
performance  and  the  individual  difference  variables  measured.  One  possible  explanation  for  the 
seeming  lack  of  consistent  findings  involves  the  nature  of  the  MT  environment  used  in  the 
various  studies.  In  the  Delbridge  study,  for  example,  participants  had  no  choice  about  when  to 
switch  tasks:  timing  was  pre-determined  by  the  experimenter.  Furthermore,  participants  were 
allowed  frequent  breaks,  and  the  tasks  were  all  very  different,  with  very  little  overlap  or  chance 
of  interference.  The  difference  between  the  findings  of  these  studies  and  the  findings  of  other 
studies  (e.g.,  Conte  &  Jacobs,  2003;  Gonzales,  2004;  Heil,  1999;  King,  Retzlaff,  Detwiler, 
Schroeder  &  Broach,  2003;  Miyake,  Friedman,  Emerson,  Witzki,  Howerter,  and  Wager,  2000; 
Suss,  Oberauer,  Wittman,  Wilhelm,  and  Schulze,  2002)  suggest  that  that  perhaps  in  order  for 
individual  differences  to  play  a  role  in  MT  performance,  the  MT  environments  need  to  either 
elicit  a  certain  level  of  stress  and/or  emphasize  specific  demands. 

The  correlational  studies  examining  the  effect  of  individual  differences  on  MT 
performance  provided  insight  into  traits  that  are  desirable  across  a  wide  variety  of  MT 
environments.  Those  who  score  higher  on  assessments  of  fluid  intelligence,  processing  speed, 
working  memory,  are  likely  to  perform  better  in  MT  environments  (Jemeic  &  Sverko,  1994; 
Cepeda,  Kramer  &  Gonzales  de  Sather,  2001;  Suss,  Oberauer,  Wittmann,  Wilhelm  &  Schultze, 
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2002;  Gonzales,  2004)),  but  the  question  still  remains  -  will  those  people  perform  equally  well  in 
all  types  of  MT  environments?  That  is,  might  some  individuals  who  score  high  on  a  particular 
individual  difference  measure  perform  well  in  some  MT  environments,  but  not  others? 

A  few  studies  examined  the  interaction  between  individual  differences  and  characteristics 
of  the  environment.  Of  these,  most  focused  on  how  personality  traits  interacted  with  differences 
in  MT  environments.  One  such  set  of  studies  (Ishizaka,  Marshall,  &  Conte,  2001;  De  la  Casa, 
Gordillo,  Mejias,  Rengel,  &  Romero,  1998)  looked  at  the  interaction  between  the  personality 
dimension,  Type  A  Behavior  Pattern  (TABP),  and  explicitness  of  task  prioritization.  Participants 
were  given  a  series  of  tests  designed  to  assess  TABP.  They  were  then  given  multiple  tasks  to 
perform,  such  as  memorization,  visual  search,  quantitative  comparisons,  and  gauge  monitoring. 
In  some  of  the  conditions',  task  prioritization  was  explicit  -  participants  were  told  which  tasks 
were  primary  and  which  were  secondary.  In  other  conditions,  they  were  told  nothing  about  the 
relative  importance  of  the  tasks.  The  primary  findings  of  these  studies  revealed  a  distinct 
interaction  between  TABP  and  the  explicitness  of  task  prioritization:  In  environments  where  it 
was  not  clear  which  of  two  tasks  takes  priority,  those  classified  as  Type  A  tended  to  be 
“hypervigilant”  and  divide  their  attention  equally  between  the  two  tasks,  while  those  not 
classified  as  having  TABP  tended  to  focus  on  one  task  more  than  another.  In  situations  where  the 
primary  task  was  clearly  defined,  however,  those  classified  as  Type  A  were  better  able  to  direct 
attention  toward  the  relevant  primary  task  and  not  be  distracted  by  the  secondary  task.  Based  on 
results  of  these  studies,  the  researchers  suggested  that  some  individuals  may  be  better  suited  to 
work  in  MT  environments  where  task  prioritization  is  relatively  stable,  while  others  may  be 
better  suited  to  work  in  environments  where  task  prioritization  is  constantly  shifting  or  may 
suddenly  change. 

A  handful  of  other  studies  looked  at  more  cognitive  individual  difference  variables,  such  as 
the  study  conducted  by  Cepeda,  Kramer,  Gonzales  de  Sather,  (2001).  This  experiment  looked  at 
how  individual  difference  factors  such  as  working  memory  capacity,  processing  speed,  and  age 
interacted  with  MT  performance  involving  tasks  with  varying  levels  of  interference  potential 
(i.e.,  tasks  that  shared  varying  degrees  of  similarity).  Findings  indicated  that  age  was  a  primary 
factor,  but  working  memory  capacity  and  processing  speed  also  explained  some  of  the  variance. 
The  differences  were  more  marked  as  task  interference  increased. 

A  relevant  issue  addressed  in  several  studies  concerns  the  stability  of  the  interaction 
between  individual  differences  and  environmental  characteristics.  Is  the  correlation  strong  during 
initial  training,  but  becomes  less  relevant  as  a  person  receives  more  training  and  becomes  more 
familiar  with  the  tasks?  Ackerman  (1992)  addressed  this  question  in  a  study  in  which  he  looked 
at  how  cognitive  ability,  perceptual  speed,  and  psychomotor  ability  predict  performance  on  MT 
tasks  that  have  either  consistent  or  inconsistent  (i.e.,  predictable  or  unpredictable)  processing 
demands.  Results  indicated  that  during  the  initial  learning  phase,  the  correlation  between  the 
individual  difference  variables  and  performance  was  high  for  both  consistent  and  inconsistent 
tasks,  but  that  as  people  became  more  familiar  with  consistent  tasks,  the  correlation  became 
attenuated.  This  indicates  that  some  people  may  do  well  in  MT  environments  when  the  rules  and 
procedures  are  fairly  predictable  and  the  information  processing  demands  are  consistent,  but  then 
have  difficulty  -  even  after  hours  of  practice  and  training  -  in  MT  environments  where  some  or 
most  of  the  tasks  are  not  predictable. 
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Along  similar  lines,  Damos  and  Smist  (1982)  looked  at  the  different  strategies  people  used 
to  manage  the  flow  of  tasks  in  a  dual  task  laboratory  study  requiring  speeded  responses.  Not 
surprisingly,  they  found  that  there  were  clear  differences  in  the  types  of  strategies  that 
individuals  used,  with  some  strategies  being  far  more  efficient  and  effective  than  others.  Those 
who  did  well  on  the  task  tended  to  first  analyze  the  environment  and  set  up  response  patterns 
enabling  them  to  offload  some  of  the  cognitive  workload  demands  and  quickly  switch  between 
tasks.  Others  tended  to  use  a  more  “massed”  approach,  focusing  on  one  task  at  a  time  while 
ignoring  the  other  task  for  long  periods  of  time.  What  was  interesting,  though,  was  that  even 
when  those  who  used  the  less  efficient  “massed”  strategy  were  asked  to  adopt  the  more  efficient 
strategy,  they  were,  for  the  most  part,  unable  to  do  so.  Therefore,  it  appears  that  performance  in 
the  MT  environment  was  not  due  solely  to  training  or  choice  of  strategy,  but  to  an  underlying 
ability. 

Based  on  the  review,  a  list  of  cognitive  and  personality  variables  was  distilled  that  have 
been  shown  to  effect  performance  in  MT  environments  and  appear  to  have  the  most  practical 
value.  The  variables  on  the  list  shown  in  Table  1  have  been  linked  to  specific  environmental 
demands,  and  other  researchers  have  identified  their  relationship  to  performance  in  multi-tasking 
conditions. 


Table  1: 

List  of  Key  Individual  Differences  Related  to  MT  Performance 

Cognitive  Variables  Personality 


attention  allocation  strategy 
baseline  arousal  levels 
ability  to  coordinate  information 
divided  attention 
fluid  intelligence 

inhibition  (suppressing  responses) 

interval  timing  ability 

managing  large  sets  of  goals 

mental  set  switching  speed 

motor  response  speed 

perceptual  accuracy  &  discrimination 

perceptual  processing  speed 

planning 

prioritization 

prospective  memory 

reasoning  about  abstract  concepts 

recognizing  abstract  relationships 

retrospective  memory 

selective  attention 

situational  awareness 

working  memory  capacity  &  updating 


complacency  potential 
conscientiousness 
coping  style 
decisiveness 
impulsivity 
locus  of  control 
mastery  orientation 
openness  to  experience 
organization 
performance  orientation 
risk  taking 

tolerance  for  high  intensity  activities 
tolerance  of  ambiguity 
trait  anxiety 

Type  A  Behavior  Pattern  factors 

-  Achievement  strivings 

-  Impatience/irritability 

-  Polychronicity 

-  Sense  of  time  urgency 
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Table  2: 

List  of  Key  Environmental  Variables  Related  to  MT  Performance 

I.  Environmental  Variables 

A.  Number  of  tasks  performed  in  a  given  time  period  (few  vs.  many) 

B.  Number  of  information  sources  monitored  in  a  given  time  period  (few  vs.  many) 

C.  Frequency  of  interruptions  (few  vs.  many) 

II.  Task  Coordination  Variables 

D.  Explicit  Priorities  vs.  Ambiguous  Priorities  (whether  the  relative  priorities  of  the  tasks 
-  which  tasks  are  primary,  which  are  secondary  -  is  pre-established  or  if  the  worker 
must  establish  priorities) 

E.  Cued  vs.  Self-Determined  Task  Switching  (whether  when  to  switch  to  a  new  task  is 
determined  or  signaled  by  the  environment  or  is  at  the  discretion  of  the  worker) 

F.  Rapid  vs.  Relaxed  Task  Switching  (switching  every  few  seconds  or  longer  time 
periods) 

G.  Erratic  vs.  Consistent  Task  Switching  (whether  pacing  is  fairly  steady  or  if  there  are 
lulls  interspersed  with  periods  of  high  intensity) 

H.  High  vs.  Low  Similarity  Among  Tasks  (whether  tasks  share  characteristics  that  might 
be  easily  confused,  or  are  quite  distinct) 

I.  Dependence  or  Coordination  vs.  Independence  of  Tasks  (whether  the  ability  to  do  one 
task  depends  on  completion  of  another  or  if  the  tasks  are  independent  from  one 
another) 

III.  Task  Variables 

J.  Short  vs.  Long  Average  Task  Duration  (whether  tasks  can  be  completed  within 
seconds  or  if  they  take  hours  to  perform) 

K.  Little  vs.  Extensive  Training  Required 

L.  Automatic  vs.  Choice  Decision  Making  decisions  have  prescribed  responses  that 
become  automatic  with  experience  or  have  multiple  potential  solutions  that  require 
choice  decisions.) 

M.  Severe  vs.  Little  Consequence  of  Failure  (On  one  end  of  this  dimension  there  is  little 
or  no  consequence  to  failing  to  perform  the  tasks  correctly  or  at  all.  On  the  other  end 
of  the  dimension,  there  are  extreme  consequences) 

N.  Speed  vs.  Accuracy  in  Performance  of  Tasks  (On  one  end  of  this  dimension,  tasks 
must  be  performed  at  the  highest  speeds  possible.  On  the  other  end  of  the  dimension, 
all  tasks  must  be  performed  at  high  levels  of  accuracy.) 

O.  Multiple  Different  Tasks  vs.  Multiple  Similar  Tasks  (whether  the  different  tasks 
require  multiple  kinds  of  skills  -  e.g.  communication,  social,  perceptual  motor, 
cognitive,  etc  —  or  the  tasks  require  similar  and  a  limited  set  of  skills 

P.  Constant  vs.  Periodic  Monitoring  Required  (whether  constant  monitoring  is  required 
or  if  the  monitoring  task  may  be  performed  periodically.) 

Q.  Type  of  Task  (such  visual  or  auditory  monitoring,  motor  response,  choice  reaction, 
etc.) 
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As  noted  previously,  the  literature  review  also  included  sources  that  described  the  effects 
of  environmental  variation  on  MT  performance.  Based  on  the  review,  a  list  was  compiled  of 
dimensions  upon  which  MT  environments  likely  vary.  The  compilation  focused  on  the  variables 
that  other  basic  and  applied  researchers  have  investigated,  that  seem  the  most  important 
characteristics  of  MT  environments,  and  that  seem  the  most  relevant  to  potential  individual 
differences.  Table  2  organizes  the  environmental  variables  in  three  categories:  those  that 
characterize  the  environment  itself,  those  that  distinguish  how  tasks  must  be  coordinated,  and 
those  that  describe  the  tasks  themselves. 

The  lists  of  individual  and  environmental  variables  extracted  from  the  literature  were  not 
sufficient  to  develop  a  model  of  MT  environmental  variation,  nor  link  that  variation  to  individual 
differences  in  abilities.  The-  literature  on  this  topic  is  fragmented  and  researchers  have  not  yet 
developed  a  comprehensive  model  of  MT  environments,  nor  have  they  attempted  to  determine 
the  relative  importance  of  individual  difference  variables.  The  literature  review  did,  however, 
provide  an  initial  framework  on  which  to  base  further  study  of  the  factors  that  affect  MT 
performance.  Based  on  research  findings,  the  lists  afforded  the  development  of  empirically 
supported  hypotheses  about  how  different  environments  and  different  workers  vary.  The  lists 
were  also  used,  in  part,  to  design  a  study  in  which  interviews  were  conducted  with  SMEs  who 
work  in  various  MT  and  non-MT  environments.  Specifically,  the  lists  were  used  to  guide  the 
development  of  an  interview  instrument  that  was  administered  via  telephone  interviews  with 
subject  matter  experts  in  a  selected  set  of  Army  MOS’s.  The  following  section  describes  the 
methods  and  results  of  that  study. 


STUDY  OF  VARIATION  AMONG  MT  ENVIRONMENTS _ 

The  purpose  of  the  interview  study  was  to  better  understand  the  variation  among  MT  envi¬ 
ronments,  how  MT  environments  differ  from  non-MT  environments,  and  how  individual  differ¬ 
ences  affect  performance  in  MT  environments.  While  the  literature  review  was  useful  in  gener¬ 
ating  hypotheses,  it  was  necessary  to  take  the  next  step,  which  was  to  explicitly  investigate 
variation  among  environments  thought  to  demand  MT  to  varying  degrees.  The  first  step  of  the 
study  was  to  select  a  set  of  Army  MOSs  that  covered  a  wide  range  of  domains  from  health  care 
to  food  services,  and  also  appeared  to  entail  differing  levels  of  MT.  Selection  was  initially  based 
on  brief  written  descriptions  found  in  Army  literature  (US  Department  of  Defense,  1993; 
http://www.goarmy.com/JobCatList.do?fw=careerindex  and  http://usmilitary.about.com/od/ 
enli  stedj  obs/a/arj  obs .  htm) . 

One  of  the  primary  purposes  of  the  interview  study  was  to  gather  detailed  descriptions  of 
each  MOS,  particularly  in  terms  of  the  dimensions  that  the  MT  literature  indicated  played  a  role 
in  effecting  differences  in  MT  performance.  For  example,  the  interviews  addressed  issues 
concerning  prioritization  demands,  task-switching  frequency,  and  level  of  decision-making 
autonomy  that  characterized  the  MOS’s.  The  goal  was  to  compile  the  information  in  a  format  in 
which  similarities  and  differences  across  the  various  MOS’s  could  be  revealed.  The  intent  was  to 
discern  any  patterns  of  variation  that  should  be  included  in  a  model  of  real-world  MT 
environments. 
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Method 

Participants.  Ten  individuals  were  recruited  as  participants  for  the  interviews.  Each 
individual  was  highly  experienced  in  one  of  ten  MOSs  that  were  selected  for  analysis:  Air 
Traffic  Controller  (15Q),  Communications  Locator/Interceptor  (98H),  Electronic  Intelligence 
Interceptor/ Analyst  (98 J),  Fire  Support  Specialist  (13F),  Food  Service  Operations  (92G),  Health 
Care  Specialist  (91W),  Multichannel  Communication  Systems  Operator  (25Q),  Radiology 
Specialist  (9 IP),  Transportation  Management  Coordinator  (88N),  and  Topographical  Analyst 
(21U).  Based  on  recommendations  of  experienced  Army  Personnel,  these  MOS’s  were  selected 
because  they  appeared  to  represent  a  wide  range  of  domains,  as  well  as  varying  levels  of  MT. 
Several  participants  were  recruited  upon  the  recommendation  of  personnel  from  the  U.S.  Army 
Human  Resources  Command.  Others  were  recruited  from  previously  established  contacts.  A 
total  of  ten  SMEs  were  interviewed.  All  of  the  participants  were  SFCs,  MSGs,  or  SMGs  and  had 
several  years  of  experience  performing  and/or  supervising  in  their  particular  MOS.  Many  of 
them  were  currently  serving  as  professional  development  coordinators. 

Materials.  Drawing  upon  the  environmental  factors  identified  above,  which  the  literature 
review  and  our  own  previous  analysis  of  MT  environments  (Fischer,  Morrin,  &  Joslyn,  2003) 
had  shown  to  be  potentially  useful,  an  interview  instrument  was  created  to  tap  into  the 
environmental  characteristics  of  each  MOS,  as  well  as  the  MT  demands  and  the  individual 
difference  traits  of  individuals  who  functioned  well  in  that  particular  environment.  Appendix  A 
contains  the  survey  instrument  used  in  the  study.  As  can  be  seen  in  Appendix  A,  the  interview 
instrument  included  quantitative  questions,  such  as  the  number  of  tasks  performed  in  a  given 
time  period;  rating  questions,  such  as  rating  on  a  scale  of  one  to  five  how  explicit  task 
prioritization  is,  with  one  being  very  explicit  and  five  being  not  at  all  explicit;  and  open-ended 
questions  such  as,  “What  distinguishes  a  top  performer  from  one  who  has  difficulty?”  Each 
question  also  included  examples,  prompts,  and  follow-up  questions  that  the  interviewer  could  use 
when  necessary. 

Procedure.  Individual  30-50  minute  phone  interviews  were  conducted  with  each  SME 
participant.  The  interviewer  first  described  the  purpose  of  the  study,  then  asked  the  participant 
about  his  or  her  background  in  the  MOS.  The  interviewer  also  asked  the  participant  how  he  or 
she  would  define  multi-tasking  to  ensure  they  understood  what  was  meant  by  the  term  as  it  was 
used  in  the  survey.  The  interviewer  then  asked  a  series  of  questions,  contained  in  the  interview 
instrument,  addressing  such  factors  as  number  and  type  of  tasks,  number  and  types  of  sources  of 
information,  memory  demands,  task  switching  frequency,  task  switching  initiation,  task 
continuity,  task  prioritization,  level  of  decision  making,  level  of  decision-making  autonomy, 
degree  of  task  urgency,  consequences  of  failure,  and  characteristics  of  individuals  who  do  well  or 
poorly  in  that  environment.  The  interviews  were  recorded  on  audio  tape. 

Results 

The  interview  responses  were  summarized  and  recorded  in  several  tables,  which  are 
contained  in  Appendix  B.  The  tables  show  a  great  deal  of  variability  among  the  ten  MOS’s  on 
most  dimensions.  That  said,  patterns  did  emerge  from  the  data.  Some  MOS’s  appeared  to 
emphasize  certain  environmental  and  individual  difference  characteristics  more  than  others.  For 
example,  in  MOS’s  where  the  environment  was  often  unpredictable  and  task  prioritization 
somewhat  flexible,  traits  such  as  ability  to  think  fast,  make  independent  decisions,  take  initiative, 
and  learn  quickly  were  frequently  mentioned.  In  more  routine  environments  where  task 
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prioritization  was  usually  pre-established,  traits  such  as  time  management  and  organizational 
skills  were  more  frequently  mentioned.  There  was,  of  course,  overlap  in  the  SMEs’  responses. 
The  ability  to  “think  on  one’s  feet,”  for  example,  is  desirable  across  all  jobs. 

Patterns  also  emerged  from  the  information  that  respondents  emphasized  as  most  important 
to  their  working  environment.  Three  different  dimensions  were  most  salient  in  the  data.  First, 
there  appeared  to  be  differences  among  environments  in  terms  of  the  type  of  multitasking 
required.  Some  environments  appeared  to  primarily  focus  on  rapid  assessment  and  decision¬ 
making  in  non-routine  situations.  An  example  of  this  might  be  the  Fire  Support  Specialist  MOS, 
where  Soldiers  have  to,  among  other  things,  gather  and  coordinate  information  to  determine  what 
is  the  best  action  to  take,  as  well  as  when,  where  and  how  to  best  carry  out  that  action.  Although 
there  are  usually  established  procedures  with  which  the  Soldiers  are  well-trained,  the 
environment  is  often  unpredictable  and  ill-defined;  this  requires  the  Soldier  to  quickly  assess  the 
situation,  weigh  the  relative  values  of  various  options,  and  then  select  the  best  possible  plan  of 
action. 

In  contrast,  other  environments  seemed  to  primarily  involve  maintaining  task  flow.  A  good 
example  of  this  type  of  environment  was  the  Food  Service  Operations  MOS,  where  there  are 
several  routine  ongoing  tasks  or  “projects”  occurring  at  the  same  time.  Although  the  Soldier 
must  still  be  able  to  handle  unexpected  events,  typically  the  environment  is  predictable  and  well- 
defined.  In  this  type  of  environment,  a  Soldier’s  primary  responsibilities  include  initiating  tasks 
at  appropriate  times,  frequently  checking  on  the  progress  of  the  tasks  or  projects,  and 
coordinating  the  timing  of  task  execution. 

A  third  type  of  MT  environment  appeared  to  involve  monitoring,  coordinating,  and 
responding  to  information  flow.  SMEs  who  described  this  kind  of  environment  frequently  talked 
about  the  integration  of  information.  A  good  example  of  this  kind  of  environment  was  the 
Electronic  Intelligence  Interceptor/ Analyst  MOS,  where  a  Soldier  must  monitor  multiple  sources 
of  information,  learn  to  allocate  their  attention  appropriately,  and  learn  to  recognize  patterns. 

The  three  types  of  environments  appeared  to  place  differing  emphasis  on  the  ability  to 
prioritize,  make  decisions,  and  manage  time.  They  also  appeared  to  vary  in  terms  of  how  much 
autonomy  the  Soldier  has  in  a  given  situation,  and  how  much  cueing  the  environment  provides. 
Hence,  there  appears  to  be  a  set  of  environmental  dimensions  that  distinguish  environments  in 
ways  that  place  different  demands  on  individual  capabilities  and  performance. 

A  second  dimension  that  appeared  to  distinguish  MT  work  settings  was  intensity  or  pacing. 
Jobs  seem  to  vary  in  terms  of  how  many  tasks  must  be  performed  in  a  given  time  period,  which 
may  be  measured  by  the  frequency  of  task  switching  and  interruptions.  In  some  situations,  the 
Soldiers  may  switch  tasks  every  few  seconds;  in  others,  it  may  be  every  few  minutes  -  or  longer. 
Some  have  time  to  at  least  partly  finish  one  task  before  moving  on  to  another,  while  others  are 
frequently  required  to  drop  and  pick  up  tasks  midstream. 

A  third  dimension  upon  which  MT  environments  appeared  to  vary  was  the  severity  of 
consequences  of  failure  to  perform  the  job  correctly.  In  some  MOS’s,  such  as  Health  Care 
Specialist,  Air  Traffic  Control  Operator,  and  Fire  Support  Specialists,  failure  to  successfully 
multi-task  could  result  in  loss  of  human  life.  In  other  MOS’s,  the  normal  and  expected 
consequences  of  failure  are  less  severe  -  such  as  loss  of  information  or  material  resources, 
damage  to  equipment,  or  inconvenience  to  others.  While  it  is  true  that  these  less  severe 
consequences  could  also  conceivably  lead  to  loss  of  life  (e.g.,  the  lost  information  might  have 
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been  critical  in  preventing  an  attack),  the  consequences  are  less  immediate  and  can  sometimes  be 
ameliorated.  Variation  in  consequences  may  have  significant  effects  on  individual  MT 
performance.  For  example,  a  particular  person  might  be  excellent  at  MT  -  unless  lives  are  at 
stake,  at  which  point  the  level  of  stress  might  inhibit  performance.  Preoccupation  with  the 
consequences  of  failure  might  demand  extensive  cognitive  resources  such  that  it  impedes  the 
ability  to  multitask. 

A  MODEL  OF  MULTI-TASKING  ENVIRONMENTS _ 

Information  that  was  gathered  from  the  literature  review,  SME  interviews,  and  our  own 
previous  research  was  used  to  create  a  model  of  how  MT  environments  differ  from  one  another, 
and  how  they  differ  from  non-MT  environments.  The  relative  contributions  of  these  three 
sources  of  inspiration  for  the  model  cannot  be  quantified.  However,  the  information  extracted 
from  the  literature  review  of  MT  environmental  variation  was  far  more  productive  and  had 
greater  influence  on  the  model  than  did  information  extracted  from  the  literature  on  individual 
differences.  Manipulation  and  comparison  of  the  effects  of  environmental  conditions  ultimately 
provided  a  framework  on  which  modeling  efforts  could  be  based.  In  contrast,  the  literature  on 
individual  differences  was  less  useful,  largely  because  it  is  fragmented.  Studies  have  typically 
focused  on  single  personality  or  preference  variables  and  have  not  attempted  to  relate  these 
variables  to  other  kinds  of  factors,  i.e.,  environmental  factors.  No  comprehensive  model  of  MT 
ability  has  been  developed.  In  short,  the  individual  difference  literature  was  not  as  helpful  to  the 
model  building  effort  as  initially  expected.  In  contrast,  study  of  environmental  variation  (from 
the  literature  and  from  the  SME  interviews)  produced  results  that  could  be  immediately  applied 
to  modeling  objectives. 

The  literature  on  individual  differences  also  leads  to  an  impractical  measurement  approach 
for  assessing  MT  ability.  The  fragmented  nature  of  the  individual  difference  literature  lends  itself 
to  an  unfeasible  assessment  strategy  in  which  a  battery  of  tests  is  used,  each  test  specific  to  one 
individual  difference  factor.  A  battery  of  tests  would  most  certainly  take  several  hours  to 
administer,  which  would  be  costly  and  unworkable  for  most  applications.  In  contrast,  the 
literature  concerning  environmental  differences  between  laboratory  and  real-world  MT 
conditions  leads  to  a  much  more  pragmatic  testing  approach  in  which  key  environmental 
variables  that  draw  out  the  important  individual  differences  are  simulated.  Such  an  approach,  of 
course,  requires  a  theoretically  grounded  and  empirically  tested  model  of  environmental 
variation  among  MT  settings,  which  must  also  be  related  to  key  individual  difference  variables 
that  determine  performance  in  those  settings.  Hence,  a  goal  of  this  research  was  to  develop  just 
such  a  model. 

Only  a  few  researchers  have  attempted  to  define  real  world  MT  environments  or 
distinguish  them  from  other  kinds  of  work  settings  (Burgess,  1998,  2000;  Joslyn  &  Hunt,  1998; 
Fischer,  Morrin,  &  Joslyn,  2003).  The  few  theoretical  and  empirical  studies  that  can  be  found  in 
the  literature  have  focused  on  the  commonalties  among  real  world  MT  environments  and  have 
offered  definitions  of  these  types  of  work  settings.  For  example,  Fischer,  Morrin,  and  Joslyn 
(2003)  proposed  that  MT  work  environments  share  eleven  characteristics:  they  (1)  have  multiple 
discrete  tasks,  (2)  require  that  tasks  must  be  interleaved  and  cannot  be  performed  simultaneously, 
(3)  require  that  tasks  cannot  be  shed  or  significantly  postponed,  (4)  do  not  signal  or  cue  task 
initiation,  (5)  are  dynamic  and  include  interruptions,  (6)  present  tasks  that  differ  in  priority. 
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difficulty,  and  time  needed  for  completion,  (7)  do  not  provide  feedback  for  some  tasks,  (8) 
require  that  tasks  be  performed  in  seconds  to  minutes,  (9)  are  time  pressured,  (10)  include  tasks 
that  demand  different  kinds  and  amounts  of  cognitive  resources,  and  (11)  require  extensive 
training  or  education.  The  model  presented  in  this  report  extends  previous  conceptions  of  MT 
environments  by  describing  how  they  vary  from  one  another  in  ways  that  are  important  to 
individual  differences  among  the  workers  who  perform  these  jobs. 

The  model  also  clarifies  previous  researchers’  (Burgess,  1998,  2000;  Fischer,  Morrin,  & 
Joslyn,  2003)  conceptions  of  how  MT  environments  are  different  from  non-MT  work  settings. 
Specifically,  the  model  states  that  MT  work  environments  should  be  distinguished  from  jobs  that 
are  simply  fast-paced.  Work  environments  frequently  require  personnel  to  perform  more  than 
one  task  within  a  limited  period  of  time.  In  some  jobs,  workers  have  to  quickly  complete  one 
task  before  they  turn  to  another.  The  next  task  must  then  be  swiftly  completed  before  yet  another 
task  is  started,  and  so  on.  The  requirement  to  sequentially  complete  tasks  under  time  constraints 
creates  a  fast-paced  environment.  However,  a  useful  distinction  can  be  made  between  work 
environments  that  require  quick  sequential  completion  of  tasks  and  those  that  are  also  fast-paced, 
but  require  task  execution  to  be  interleaved.  The  requirement  to  interleave  multiple  tasks  under 
time  constraints  has  been  identified  as  a  key  distinguishing  characteristic  of  multi-tasking  (MT) 
environments  (Burgess,  1998,  2000;  Fischer,  Morrin,  &  Joslyn,  2003).  This  distinction  is 
important  because  the  demand  to  interleave  tasks  requires  different  cognitive  capabilities  than 
does  the  demand  to  sequentially  complete  tasks.  For  example,  heavy  demands  are  placed  on 
workers’  prospective,  short-term,  and  working  memory  when  interleaving  tasks  because  they 
must  remember  to  reinitiate  tasks  that  have  been  temporarily  suspended. 

The  model  proposes  that  the  variance  among  MT  environments  is  multidimensional.  That 
is,  real  world  environments  vary  in  many  ways,  along  many  different  factors.  Some  dimensions 
on  which  MT  environments  vary  have  the  result  of  placing  different  demands  on  workers,  while 
other  dimensions  do  not.  For  example,  the  kind  of  tasks  the  worker  is  required  to  perform  varies 
among  MT  environments.  Some  environments  present  workers  with  ill-defined  problem  spaces, 
within  which  they  must  make  rapid  decisions.  Others  present  problems  that  have  routine 
solutions,  but  require  workers  to  perform  multiple  interleaving  actions.  Figure  2  shows  a  two- 
dimensional  representation  of  several  environmental  dimensions  that  the  present  research 
suggests  are  important  and  influential  factors  that  characterize  real-world  MT  environments. 
These  differences  probably  demand  varying  cognitive,  physical,  and  emotional  skills.  Figure  2  is, 
of  course,  somewhat  misleading  because  of  the  necessary  reduction  to  a  planar  view. 

As  noted  previously,  the  present  research  suggests  that  key  environmental  variables  of  MT 
work  settings  co-vary  in  systematic  ways.  For  example,  consider  the  following  environmental 
dimensions. 

•  ill-defined  problem  space  vs.  well-defined  problem  space 

•  multiple  potential  solutions  vs.  single  potential  solution 

•  freedom  to  prioritize  tasks  vs.  environmentally  determined  prioritization 

•  autonomous  task  switching  vs.  environmental  cued  task  switching 

The  results  of  the  interviews  conducted  in  the  present  research  suggest  that  these 
dimensions  cluster  to  form  different  types  or  kinds  of  environments.  For  example,  environments 
that  present  ill-defined  problem  spaces  also  typically  require  the  worker  to  prioritize  tasks  and 
self-determine  when  they  will  switch  to  another  task.  These  types  of  environments  typically 
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place  heavy  demands  on  situation  assessment  and  rapid  decision-making  capability.  Conversely, 
other  jobs  that  present  problem  spaces  that  are  well  defined  and  have  only  one  acceptable 
solution  also  tend  to  provide  inviolable  prioritization  of  tasks,  and  task  switching  is  often 
environmentally  cued.  The  effect  of  the  covariation  among  the  multiple  dimensions  on  which 
environments  vary  is  to  create  types  of  MT  environments.  Therefore,  it  is  possible  to  derive  a 
typology  of  MT  environments  that  captures  many  important  differences  among  the 
environments,  particularly  differences  that  place  different  kinds  of  demands  on  workers. 


Figure  2.  Multidimensional  space  characterizing  variation  in  MT  environments. 

There  are  probably  a  smaller  set  of  key  variables  or  sets  of  variables  that  have  the  greatest 
effect  on  the  kinds  of  capabilities  workers  must  have  in  order  to  perform  well  in  the 
environment.  One  of  these  is  the  consequence  of  error  or  failure.  While  no  one  likes  to  fail, 
failure  in  some  environments  is  less  consequential  (e.g.,  food  preparation)  than  in  others  (e.g., 
nursing,  artillery  fire  support,  or  air  traffic  control).  Variation  in  the  severity  of  consequence 
among  MT  environments  probably  affects  worker-perceived  stress  levels,  willingness  to  perform 
the  job,  and  attrition.  Hence,  variability  among  environments  in  terms  of  the  consequence  of 
failure  probably  interacts  with  certain  individual  characteristics.  Environments  that  present  high 
consequences  demand  the  worker  be  at  least  emotionally  stable,  resistant  to  stress,  and  confident, 
among  other  individual  characteristics. 

As  noted  previously,  a  second  dimension  that  distinguishes  MT  environments  is  the 
intensity  of  multi-tasking  required.  Some  MT  environments  simply  require  more  tasks  per  unit 
time,  are  faster  paced,  or  present  more  uncontrollable  interruptions.  The  intensity  of  the 
environment  probably  influences  the  required  MT  skill  level  a  worker  needs.  Individuals  with 
moderate  MT  abilities  may  do  well  at  moderate  intensity  jobs,  but  are  likely  to  perform  poorly  at 
jobs  that  are  at  the  highest  levels  of  intensity.  Again,  variation  in  intensity  most  likely  covaries 
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with  certain  individual  differences  to  affect  overall  performance.  Workers  in  high  intensity 
environments  probably  have  to  have  a  large  working  memory  capacity,  excellent  organization 
skills,  good  prospective  and  short-term  memory,  and  good  logic  and  reasoning  skills,  for 
example. 

A  set  of  environmental  variables  that  involve  the  kind  of  tasks  required  by  the  environment 
constitutes  another  group  of  important  dimensions.  These  include  complex  vs.  routine  decision 
making;  freedom  and  responsibility  to  prioritize  vs.  prescribed  prioritization;  ill-defined 
problems  with  multiple  solutions  vs.  well-defined  problems  with  only  one  or  few  solutions  that 
satisfy  constraints;  and  autonomous  vs.  environmentally  cued  or  forced  task  switching.  As  noted 
previously,  particular  values  of  these  dimensions  most  likely  co-occur  in  real-world  MT 
environment,  which  has'  permitted  us  to  identify  three  general  types  or  kinds  of  MT 
environments.  Based  on  the  likely  covariation  of  these  variables,  an  initially  simple,  but  testable, 
typology  of  MT  environments  is  offered. 

The  first  kind  of  environment  described  by  the  model  requires  high  levels  of  decision¬ 
making  capability.  Environments  that  tend  to  require  complex  decision-making  also  tend  to 
provide  freedom  and  responsibility  in  prioritizing.  Such  environments  also  tend  to  present  ill- 
defined  problems  for  which  there  may  be  multiple  solutions  that  satisfy  constraints.  The  central 
task  in  such  environments  may  be  to  allocate  limited  resources.  Company  commander  and  fire 
support  specialist  (field  artillery)  are  two  good  examples  of  this  kind  of  MT  position. 

The  second  kind  of  environment  presents  workers  with  well-defined  problems  that  require 
routine  decision-making.  In  such  environments,  the  worker  learns  a  set  of  “canned”  solutions  to 
the  problem.  Task  switching  is  often  cued  by  the  environment  in  these  jobs,  and  hence, 
prioritization  of  tasks  is  well  defined.  The  main  MT  issue  in  this  second  kind  is  to  keep  all  of  the 
many  tasks  going  and  manage  the  task  flow  -  or  keep  all  the  balls  in  the  air,  if  you  will.  Some 
nursing  positions  (e.g.,  floor  nurse  in  some  hospital  departments)  and  food  service  positions  are 
examples  of  this  kind  of  environment. 

A  third  kind  of  MT  environment  identified  by  the  model  is  characterized  by  its  multiple 
sources  of  information.  The  sources  may  provide  information  in  auditory,  visual,  or  tactile 
mediums.  The  worker’s  job  in  this  third  type  of  environment  is  to  monitor  and  respond  to 
multiple  inputs  or  to  integrate  the  various  sources  of  information  to  derive  meaning.  The 
characteristics  of  the  three  kinds  of  MT  environments  are  described  in  greater  detail  in 
Appendix  C. 

The  model  of  MT  environments,  depicted  in  Figure  3,  identifies  three  key  orthogonal 
dimensions.  Figure  3  displays  three  kinds  or  types  of  environments,  as  described  above,  which 
form  one  categorical  dimension.  The  three  types  are  shown  as  the  “pie  slices”  in  the  figure.  The 
second  dimension  identified  by  the  model  is  the  intensity  of  MT  activity.  Intensity  of  each  of  the 
three  kinds  of  environments  varies  from  low  to  high.  High  intensity  environments  are  depicted  in 
Figure  3  as  the  darker  saturated  center  regions  of  each  circle.  The  outer  rings  of  the  circles 
represent  low  MT  intensity  environments.  It  is  important  to  note  that  the  outer  rings  of  the  model 
do  not  represent  non-MT  environments  and  the  model  does  not  attempt  to  describe  those  kinds  of 
work  settings.  The  model  is  therefore  limited  to  the  domain  of  MT;  its  boundaries  exclude  work 
domains  that  do  not  meet  the  definition  of  MT  it  proffers.  If  one  were  to  depict  non-MT 
environments  in  Figure  3,  one  would  have  to  place  them  outside  of  the  shaded  rings.  The  third 
orthogonal  variable  identified  in  the  model  is  the  consequence  of  error  or  failure  in  the 
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environment,  ranging  from  minimal  consequences  to  severe  consequences.  This  is  represented  in 
Figure  3  by  the  multiple  and  layered  circles,  with  the  circles  in  the  rear  portion  of  the  figure 
having  minimal  consequences  and  the  ones  toward  the  front  of  the  figure  having  severe 
consequences. 


Figure  3.  Depiction  of  model  of  MT  environments. 

As  will  be  discussed  in  the  measurement  approach  below,  these  three  orthogonal 
dimensions  can  be  directly  linked  to  specific  factors  that  characterize  differences  in  MT 
environments,  such  as  timing  interval  between  task,  level  of  autonomy,  and  degree  of  established 
prioritization  that,  in  turn,  have  been  shown  to  draw  out  individual  differences  in  performance. 
Thus  the  model  not  only  provides  a  structured  description  of  MT  environments,  it  can  also  be 
used  to  guide  the  design  of  tests  that  can  be  tailored  to  predict  performance  in  particular  MT 
work  environments.  In  the  following  section  of  this  report,  a  measurement  approach  for 
assessing  MT  ability  is  discussed,  which  was  developed  based  on  the  model  shown  in  Figure  3. 

A  MEASUREMENT  APPROACH  FOR  ASSESSING  MT  ABILITIES  SPECIFIC  TO 
VARYING  MT  ENVIRONMENTS _ 

The  findings  of  Tasks  1  and  2  were  used  to  guide  the  design  of  a  measurement  approach, 
which  could  be  used  to  evaluate  multitasking  ability  in  a  wide  variety  of  environments.  The 
measurement  approach  is  actually  a  relatively  complex  assessment  system,  which  we  refer  to  as 
the  Multi-Tasking  Assessment  System  (MTAS).  The  proposed  measurement  approach  not  only 
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assesses  individuals’  MT  abilities,  but  also  assesses  work  environments  to  determine  (1)  whether 
an  environment  qualifies  as  an  MT  work  setting  and  (2)  the  kinds  of  MT  demands  they  place  on 
workers.  Figure  4  shows  that  the  MTAS  consists  of  two  main  components:  the  Environment 
Assessment  Tool  (ENVAT),  which  assesses  the  MT  requirements  of  work  environments,  and  a 
test  component  that  includes  three  basic  versions  of  the  Multi-Tasking  Ability  Test  (MTAT), 
each  of  which  assesses  individuals’  performance  in  a  particular  type  of  MT  environment.  The 
testing  component  also  includes  three  sub-versions  of  each  MTAT  test,  which  are  adaptively 
administered  and  are  designed  to  tap  performance  differences  associated  with  environmental 
variation  at  three  levels  of  intensity.  Therefore,  the  system  actually  includes  nine  test 
instruments. 

The  system’s  design .  (MTAS)  is  predicated  on  the  notion  that  the  highest  predictive 
capability  can  be  obtained  by  first  assessing  the  environment  of  concern  and  the  demands  it 
places  on  individuals.  Once  the  important  characteristics  of  the  environment  are  known,  then 
individuals  can  be  tested  to  determine  whether  their  skills  match  the  demands  of  the 
environment.  Figure  4  shows  that  information  gleaned  by  the  environmental  assessment 
component  is  used  to  select  the  most  appropriate  version,  or  multiple  versions,  of  the  MTAT  to 
administer  to  individuals.  The  environmental  assessment,  combined  with  an  individual’s  MTAT 
test  results,  is  used  to  predict  how  an  individual  will  perform  in  that  particular  environment.  The 
MTAS  produces  a  variety  of  products  that  contain  needed  information  to  the  purpose  of  selection 
or  placement.  Representatives  of  the  organization  using  the  MTAS  would  first  be  provided  with 
profiles  of  the  environment  of  interest.  Figure  4  shows  that  the  ENVAT’s  output  includes  a  job 
environment  report  (see  top  right  box  of  Figure  4).  A  variety  of  scores  and  information  about  the 
performance  of  each  test  taker  would  also  be  provided  to  the  organizational  representative.  Raw 
scores,  percentile  scores,  written  synopses,  and  expectancy  charts  would  be  produced  by  the 
MTAS  to  describe  individuals’  performance  (See  lower  right  box  of  Figure  4).  Each  component 
of  the  MTAS  is  explained  in  greater  detail  below. 

Environmental  Assessment  Tool  (ENVAT) 

Assessment  of  the  multi-tasking  work  environment  would  be  conducted  using  a  brief,  web- 
based  survey  called  the  ENVAT  (see  upper  left  box  in  Figure  4),  which  must  be  completed  by  a 
test  administrator  who  is  well  experienced  with  the  environment  of  concern,  or  a  qualified 
subject  matter  expert.  The  individual  must  be  thoroughly  familiar  with  the  particular  work 
environment  and  must  be  able  to  articulate  the  job  demands  of  that  environment  using  the 
ENVAT.  The  ENVAT  would  consist  of  a  series  of  scales  corresponding  to  key  dimensions  on 
which  environments  vary.  In  combination,  the  scales  would  be  used  to  assess  what  type  of 
environment  the  job  entails,  as  well  as  the  level  of  MT  intensity.  Questions  that  relate  to  the  type 
of  MT  environment  would  focus  on  dimensions  such  as  how  well-structured  the  environment  is, 
the  explicitness  of  task  prioritization,  the  level  of  autonomy  in  decision-making,  and  the  types  of 
tasks  that  are  performed.  Questions  that  pertain  to  MT  intensity  would  focus  on  dimensions  such 
as  the  number  of  tasks  that  are  typically  carried  out  per  given  unit  of  time,  the  degree  of  urgency, 
the  number  of  sources  of  information,  task-switching  frequency,  and  so  on.  To  ensure  reliability 
and  validity  of  each  dimension,  questions  could  be  asked  in  a  variety  of  ways,  such  as  multiple 
choice,  scaling,  and  other  formats.  Table  3  provides  examples  of  question  formats  that  could  be 
utilized  in  the  ENVAT. 
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Table  3. 

Potential  Question  Formats  for  the  ENVAT 


♦  Which  of  the  following  descriptions  BEST  describes  the  general  work  environment: 

A)  Keeping  many  ongoing  activities  or  projects  running  at  the  same  time,  etc.  For 
example, ... 

♦  Which  of  the  following  jobs  is  most  similar  to  the  work  environment,  not  in  terms  of 
content,  but  in  terms  of  general  pacing,  types  of  tasks,  etc. 

A)  Job  X,  where  the  person  does . etc. 

♦  Read  the  descriptions  of  the  three  job  candidates  below.  Which  one  would  be  a  best 
fit  for  the  MOS? 

A)  Job  Candidate  A  is  good  at  X,  but  is  not  good  at  Y.  His/her  supervisor  noted. .  .etc. 
Highly  specific  questions  could  also  be  asked,  such  as: 

Think  of  a  period  where  someone  in  this  position  is  engaging  in  a  high  level  of  MT.  On 
average,  how  many  tasks  are  they  performing  at  the  same  time? 


The  questions  would  be  carefully  tested  in  order  to  ensure  that  they  are  understandable  and 
provide  useful  information  about  the  dimensions  of  a  particular  MT  environment.  Environmental 
consistency  issues  would  also  be  addressed  in  the  ENVAT.  If,  for  example,  some  environments 
vary  in  type  of  MT,  intensity,  and  even  severity  of  consequence  within  themselves,  the  ENVAT 
would  account  for  that  variation,  selecting  the  method  that  would  produce  the  most  accurate 
performance  predictions.  As  is  depicted  in  Figure  4,  once  the  environment  has  been  defined 
along  key  dimensions,  the  next  step  is  to  assess  individuals’  MT  abilities  in  order  to  determine  if 
they  match  the  demands  required  by  the  environment. 

Testing  Component  of  the  MTAS:  Assessment  ofMT  Abilities 

Figure  4  shows  that  the  test  component  of  the  proposed  MTAS  comprises  three  tests, 
purported  to  predict  performance  in  the  three  different  types  of  MT  environments  described  in 
the  model  (See  lower  left  box  of  Figure  4).  The  test  component  of  the  MTAS  would  include 
high,  medium,  and  low  intensity  versions  of  each  type  of  test.  Depending  on  input  from  the 
ENVAT  that  describes  the  requirements  of  a  candidate  environment,  one  or  more  of  the  tests 
would  be  administered  to  a  target  population  for  the  purposes  of  selection,  placement,  or  even 
general  information  to  increase  self-awareness. 

The  intensity  versions  of  the  MTAS  will  assess  how  well  an  individual  can  perform  in  MT 
environments  of  varying  intensity.  The  MT  literature,  as  well  as  comments  made  during  our 
interviews  with  SMEs,  indicates  that  intensity  or  pacing  of  the  environment  plays  a  definite  role 
in  affecting  individuals’  performance  in  a  particular  environment.  For  example,  some  individuals 
may  function  well  in  positions  such  as  a  signals  interceptor/analyst  when  the  incoming 
information  is  coming  in  at  a  moderate  pace,  but  as  the  pacing  increases,  their  performance 
begins  to  break  down.  Others,  however,  appear  to  thrive  in  fast-paced  environments,  and  are  able 
to  adjust  to  the  increasing  demands. 
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An  adaptive  approach  to  assessing  capability  associated  with  intensity  would  be 
incorporated  into  each  of  the  three  versions  described  above.  For  example,  Test  #1  might  have  a 
moderate  task  interval  of  perhaps  30  seconds.  Based  on  how  well  the  user  performed  at  that 
interval  level,  the  computer  would  then  select  the  interval  level  for  the  second  test.  Test  #2  would 
then  present  tasks  at  either  a  15  second  or  a  45  second  interval.  Test  #3  would  also  either 
increase  or  decrease  the  timing  interval  by  a  fixed  amount  depending  on  how  well  the  individual 
did  on  the  previous  test.  The  appropriate  timing  levels  and  the  criteria  for  increasing  and 
decreasing  them  would  be  determined  by  data  obtained  in  the  validation  studies.  One  advantage 
of  using  an  adaptive  approach  with  intensity  level  is  that  it  will  give  users  time  to  leam  the 
environment  at  a  slower  pace  before  moving  to  a  more  intense  level,  and  it  will  provide  greater 
flexibility  in  testing.  If  a  person  performs  quite  adequately  at  a  moderately  paced  environment, 
but  has  difficulty  performing  in  a  higher-paced  one,  then  the  assessment  will  reveal  that;  there  is 
no  need  to  have  him  or  her  sit  through  a  high-paced  version  of  the  test. 

As  previously  discussed,  we  propose  that  the  most  efficient  and  effective  way  to  assess  MT 
ability  for  multiple  environments  is  to  simulate  those  environments  in  a  performance-based 
assessment  of  multitasking,  such  as  that  used  in  the  MTAT.  This  is  in  contrast  to  a  strategy  that 
administers  a  battery  of  individual  tests  that  each  addresses  a  particular  individual  difference 
variable  associated  with  MT  performance.  The  MTAT  simulates  many  aspects  of  a  complex, 
fast-paced  MT  environment  by  requiring  users  to  simultaneously  apply  many  of  the  skills  com¬ 
monly  demanded  by  MT  environments,  such  as  allocating  attention  efficiently  while  prioritizing 
tasks,  remembering  results  of  past  actions,  and  remembering  to  carry  out  future  actions. 
Furthermore,  the  MTAT  can  be  modified  to  increase  cognitive  demands  relating  to  specific  types 
of  MT  and  to  differing  levels  of  intensity.  In  the  following  section,  we  provide  specific  informa¬ 
tion  about  how  the  MTAT  would  be  modified  to  create  a  battery  of  tests  that  can  be  used  flexibly 
to  predict  performance  in  different  MT  environments  that  vary  in  type  and/or  intensity. 

Creating  Different  Test  Types  Based  on  MTAT  Variations.  The  MTAT  environment  is 
ideal  for  simulating  variation  in  MT  environments.  Research  has  shown  that  Type  A  Behavior 
Pattern  (TABP)  is  related  to  performance  in  MT  environments,  and  variation  among 
environments.  It  turns  out  that  people  high  in  TABP  respond  differently  to  environments  that 
have  clear  task  prioritization  rules  than  to  environments  that  present  ambiguous  prioritization 
rules  (Lieberman  &  Rosenthal,  2001;  De  la  Casa,  et.  al,  1998).  Knowing  the  relationship  of 
performance  to  TABP,  one  testing  strategy  would  be  to  administer  a  test  of  TABP  to  predict 
whether  a  person  might  do  well  in  an  ambiguous  environment.  But,  instead  of  administering  a 
typical  Type  A  Behavior  Pattern  assessment  tool  as  a  measurement  approach,  such  as  the 
Framington  Type  A  Scale,  one  could  instead  create  a  version  of  the  MTAT  where  prioritization 
rules  are  ambiguous  and  another  version  where  they  are  explicit,  then  look  at  how  well 
individuals  perform  in  those  two  environments.  The  underlying  mechanism  behind  performance 
differences  may  be  due  to  a  mixture  of  Type-A  personality  type,  differences  in  attentional 
allocation  strategy,  and  inhibition.  But  rather  than  address  these  variables  individually,  it  is  far 
more  efficient  to  simulate  environments  that  draw  out  these  differences,  and  then  analyze  their 
performance  patterns. 

The  MTAT  is  essentially  a  computer-based,  abstract  simulation  of  an  MT  environment.  It 
involves  presenting  the  user  with  a  series  of  objects  that  they  must  correctly  classify  into  one  of 
four  bins  based  on  features  of  the  object  such  as  its  size,  shape,  and  color.  The  users  are  never 
shown  the  bins,  they  are  just  told  which  features  the  bins  accept  (such  as  “red  circles  of  any 
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size”,  or  “tiny,  blue,  triangles”).  They  are  also  never  shown  the  objects.  They  are  just  told, 
periodically,  that  a  particular  object  is  available.  To  complete  the  object  classification  task,  the 
user  must  perform  a  series  of  subtasks  -  mainly  initiating  queries  -  that  provide  information 
about  the  features  of  a  particular  object.  At  the  same  time,  they  must  also  note,  remember  and 
keep  track  of  which  objects  are  available  and  which  are  in  need  of  classification.  What  makes  the 
simulation  an  MT  environment  is  that  at  any  one  point  in  time,  the  user  typically  has  multiple 
objects  available  to  be  classified,  and  more  appear  on  a  regular  basis.  The  MTAT  shares  many 
features  that  are  common  to  MT  environments,  and,  as  previously  noted,  is  highly  predictive  of 
performance  in  simulations  of  MT  environments  such  as  emergency  dispatching  and  air  traffic 
control.  One  main  advantage  of  the  MTAT  is  that,  unlike  these  more  concrete  simulations,  it 
does  not  require  prior  content  knowledge,  and  the  entire  test,  including  practice  and  training, 
takes  less  than  30  minutes  rather  than  several  hours.  Thus  it  is  an  ideal  test  to  use  in  selection  and 
placement  of  personnel  into  MT  environments. 

Previous  research  analyzing  civilian  MT  environments  (Fischer,  Morrin,  &  Joslyn,  2003), 
review  of  the  MT  literature,  and  preliminary  analysis  of  our  own  interviews  with  SMEs  has 
shown  that  the  current  version  of  the  MTAT  incorporates  many  features  which  address  the 
variables  that  are  critical  to  proficient  performance  in  most  MT  environments.  Table  4  lists  some 
of  the  key  environmental  characteristics  that  are  common  to  most  MT  environments,  associated 
individual  difference  variables  that  underlie  differences  in  MT  performance,  as  well  as  features 
of  the  MTAT  that  tap  into  those  differences. 

Those  who  score  well  on  the  MTAT  would  probably  function  well  in  an  environment  that 
shares  these  characteristics,  while  those  who  score  poorly  would  probably  not  do  well.  These 
variations  in  performance  might  be  explained  by  individual  differences  in  the  cognitive  and 
personality  variables  that  are  tapped  by  specific  features  of  the  MTAT.  As  the  model  of  MT 
environments  indicates,  however,  it  appears  that  some  MT  environments  may  place  greater 
emphasis  on  certain  abilities  than  do  other  environments.  For  example,  in  an  environment  that  is 
often  non-routine,  where  task  prioritization  fluctuates,  and  there  are  multiple  solutions  or  possi¬ 
ble  actions  to  select  from,  there  is  likely  to  be  a  greater  demand  placed  on  the  ability  to  prioritize 
and  rapidly  assess  situations  than  there  would  be  in  a  more  structured  and  routine  environment. 
Therefore,  to  best  predict  performance  in  that  environment,  a  test  would  have  to  emphasize  those 
abilities.  Likewise,  some  environments  place  greater  demands  on  the  ability  to  allocate  attention 
effectively  among  several  sources  of  incoming  information.  While  the  current  version  of  the 
MTAT  does  tap  into  that  ability  (it  would  not  be  possible  to  score  well  on  the  test  if  one  were  not 
able  to  attend  to  multiple  sources  of  incoming  information),  the  demands  might  not  be  strong 
enough  to  discriminate  among  test  takers  who  score  at  the  high  end;  that  is,  differentiate  among 
those  who  are  skilled  and  those  who  are  merely  adequate,  thus  reducing  the  predictive  power  of 
the  MTAT.  For  example,  as  was  noted  earlier,  a  previous  version  of  the  MTAT,  the  ADM,  was 
able  to  explain  50%  of  the  variance  of  dispatching  simulation  performances,  but  only  25%  of  the 
variance  of  air  traffic  control  simulation  performances.  It  may  be  that  there  are  characteristics 
independent  of  MT,  such  as  spatial  ability,  that  contribute  to  differences  in  performance  on  the 
ATC  simulation,  but  the  lower  predictive  value  may  also  be  due  to  differences  in  the  nature  of 
the  MT.  In  other  words,  ADM  may  incorporate  more  of  the  demands  that  are  typical  of  emer¬ 
gency  dispatching  environments  and  fewer  of  the  demands  that  are  required  by  ATC  environ¬ 
ments,  such  as  the  ability  to  efficiently  monitor  several  sources  of  information. 
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Table  4. 

Match  of  Environmental  and  Individual  Difference  Variables  to  MTAT  Features 


Several  tasks  to  perform  within 
relatively  short  period  of  time 

Mental  set  switching; 
prioritization 

♦  alternate  between  specifying 
relevant  object  #;  querying  size,  form, 
color;  classifying  objects,  noting 
presence  of  new  objects 

Several  different  rules,  procedures 
to  learn 

Retrospective  memory  capacity; 
ability  to  quickly  learn  rules  and 
keep  them  straight 

♦  remembering  rules  of  the  game 

♦  remembering  bin  attributes 

Several  different  cases  (i.e., 
ongoing  tasks,  situations,  or 
incidents)  to  keep  track  of 

Retrospective  memory,  working 
memory,  inhibition 

♦  remembering  which  objects  have 
been  classified 

♦  not  confusing  characteristics  of 
current  object  with  that  of  previous 
object 

Interruptions:  New  cases  appear 
prior  to  the  completion  of  old  ones 

♦  working  memory 

♦  prospective  memory 

♦  selective  attention 

♦  prioritization 

♦  must  re-initiate  current  line  of 
queries  when  new  object  appears 

♦  must  remember  object  number  of 
current  case  (object  they  were 
querying) 

♦  must  remember  which  objects  still 
need  to  be  classified 

♦  decide  whether  to  continue  with 
current  object  or  attend  to  new  one 

Tasks  are  generally  continuous  (if 
interrupted,  must  pick  up  where 
one  left  off) 

♦  retrospective  memory 

♦  mental  set  switching 

♦  inhibition  of  prior 
info/responses 

♦  appearance  of  new  objects  interrupts 
flow  of  queries,  which  must  be  re¬ 
initiated 

Urgency 

♦  processing  speed 

♦  tolerance  of  stress 

♦  objects  must  be  attended  to  or  they 
lose  points. 

Task  interdependence  and 
sequence 

♦  retrospective  memory 

♦  mental  set  switching 

♦  ability  to  perform  one  task 
(assigning)  depends  on  outcome  of 
previous  task  (querying) 

Rules  and  procedures  may  change 
depending  on  situation 

Inhibition 

♦  bin  attributes  change  every  test; 
must  be  able  to  inhibit  old  rules 

Incoming  information  may  be 
fragmented;  needs  to  be  integrated 

Working  Memory 

♦  remembering  and  integrating  results 
of  queries  (i.e.,  object  attributes) 

It  therefore  appears  that  the  current  version  of  the  MTAT  may  not  be  sufficient  to  assess 
performance  potential  in  the  different  types  of  MT  environments  outlined  in  our  model,  and  thus 
will  need  to  be  modified.  One  option  is  to  increase  the  predictive  power  of  the  MTAT  by 
incorporating  additional  environmental  demands  into  the  assessment.  Our  prior  research  with  the 
MTAT,  however,  has  indicated  that  the  assessment  is  already  very  challenging  for  most  people. 
If  too  many  additional  rules  and  tasks  are  added,  it  could  overburden  the  test-taker,  resulting  in 
poor  performance  for  most  of  them,  creating  a  floor  effect,  and  reducing  the  predictive  power  of 
the  test.  Therefore,  to  increase  predictive  power,  three  different  versions  of  the  MTAT  would  be 
created,  each  corresponding  to  one  of  the  three  main  MT  environments  identified  in  the  model. 
Each  version  would  have  the  basic  structure  of  MTAT.  That  is,  each  would  address  the 
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components  that  are  common  to  all  MT  environments,  such  as  the  need  to  perform  multiple 
tasks,  fairly  fast  pacing,  remembering  to  take  future  actions,  deciding  what  action  to  take,  and  so 
on.  However,  each  version  would  vary  on  how  much  emphasis  it  places  on  the  key 
characteristics  and  corresponding  abilities  that  distinguish  the  three  MT  environment  types.  For 
example,  the  Decision-Making  environment  involves  relatively  complex  decision-making  and 
greater  autonomy  in  prioritization.  Therefore,  the  version  that  correspond  to  the  Decision- 
Making  environment  (MTAT-DM)  would  incorporate  tasks  and  rules  that  places  a  greater 
emphasis  on  the  ability  to  quickly  assess  a  situation  and  select  the  most  appropriate  solution  out 
of  many  possible  solutions.  The  version  that  corresponds  to  the  Information  Monitoring 
environment  (MTAT-IM)  would  require  that  the  user  monitor  and  respond  to  more  sources  of 
information.  The  version  that  corresponds  to  the  Task  Flow  Management  environment  (MTAT- 
TFM)  would  require  the  user  to  handle  multiple  cases  at  the  same  time  and  to  coordinate  several 
ongoing  tasks. 

Table  5  provides  examples  of  how  each  of  the  three  versions  of  MTAT  might  vary.  The 
first  column  lists  some  of  the  key  environmental  demands  that  are  particularly  emphasized  by 
each  of  the  three  MT  environment  types  outlined  in  our  model  of  MT  environments;  the  second 
column  lists  some  of  the  individual  difference  variables  that  may  underlie  performance 
differences  in  environments  that  share  those  characteristics.  Finally,  the  third  column  outlines 
some  possible  ways  the  MTAT  might  be  modified  to  tap  into  those  differences.  As  was 
previously  noted,  the  relatively  abstract  nature  of  the  MTAT  allows  it  to  be  easily  modified  to 
incorporate  additional  tasks  as  well  as  more  complex  decision-making  rules  without  altering  the 
basic  underlying  structure  or  adding  additional  training  requirements. 


Table  5. 

Environmental  Variables  associated  with  MT  Environments, 
Individual  Difference  Variables,  and  MTAT  Features 


■Kn-makfyg  $i$Iron 
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Environmental 

Variable 

Individual  Difference 
Variable 

Possible  MTAT  Feature 

Ambiguous  Prioritization 

♦  Attention  Allocation  (Type 

A  vs.  Type  B  personalities) 

♦  Tolerance  of  ambiguity 

♦  prioritization 

♦  have  two  tasks  of  varying  value; 
priority  depends  on  situation 

♦  user  can  set  prioritization 

Resource  Allocation 
Decisions 

♦  retroactive  memory 

♦  prospective  memory 

♦  cognitive  strategy 

♦  planning 

♦  reasoning 

♦  working  memory 

♦  competing  demands  for  same  resource 
(either  time,  space,  materials) 

♦  differential  value  based  on  making 
most  efficient  allocation 

Multiple  solution/action 
options 

♦  reasoning 

♦  working  memory 

♦  fluid  intelligence 

♦  provide  several  viable  options  that 
change  value  depending  on  features  of  a 
particular  situation 

Autonomy  in  task 
switching 

♦  proactive  memory 

♦  cognitive  strategy 

♦  requirement  to  initiate  new  tasks 
(rather  than  rely  on  environmental  cues) 

Unpredictable  environment 

- -  - 

♦  fluid  intelligence 

♦  rules  or  situations  change  independent 
of  the  user’s  actions 

Rapid  assessment 

♦  fluid  intelligence 

♦  situational  awareness 

♦  several  sources  of  information  that 
must  be  combined  to  make  assessment 
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Table  5  (Continued). 

Environmental  Variables  associated  with  MT  Environments, 
Individual  Difference  Variables,  and  MTAT  Features 


Environmental 

Variable 

Individual  Difference 
Variable 

Possible  MTAT  Feature 

Several  on-going  tasks 

♦  organization  skills 

♦  situational  awareness 

♦  requirement  to  switch  between 
tasks/cases  rather  than  complete  one 
before  moving  to  next 

Scheduled  Task-switching 

♦  time  management 
♦.  prospective  memory 

♦  need  to  adhere  to  established 
procedures 

Somewhat  routine  decision 
making 

♦  retrospective  memory 

♦  need  to  adhere  to  established 
prioritization/rules 

Tasks  and  cases  need  to  be 
checked  on 

♦  prospective  memory 

♦  attention  allocation 

♦  have  several  objects/cases  that  change 
over  time 

Environmental 

Variable 

Individual  Difference 
Variable 

Possible  MTAT  Feature 

Many  sources  of  incoming 
information 

♦  divided  attention 

♦  attention  allocation  strategy 

♦  have  split  screens  or  separate  screens 
with  different  information  sources  that 
need  to  be  integrated  to  successfully 
perform  a  task 

Competing  demands 

♦  selective  attention 

Information  needs  to  be 
integrated 

♦  reasoning 

_ 

Table  6  provides  examples  of  some  of  the  key  environmental  demands  that  relate  to 
variations  in  MT  intensity,  as  well  as  possible  individual  difference  variables  that  may  underlie 
disparities  in  performance  and  possible  ways  the  MTAT  might  be  modified  to  tap  into  those 
factors.  The  most  efficient  approach  is  to  use  an  adaptive  approach  whereby  factors  such  as  the 
time  interval  between  tasks  is  increased  or  decreased  as  the  test-taker  progresses  through  the  test. 
The  bottom  left  comer  of  Figure  4  outlines  how  the  adaptive  approach  might  be  implemented. 


Table  6. 

Environmental  Variables  related  to  MT  Intensity, 
Individual  Difference  Variables,  and  MTAT  Features 


Environmental  Variable 

Individual  Difference 
Variable 

Possible  MTAT  Features 

Rapid  vs.  relaxed  pacing 

♦  perceptual  speed 

♦  tolerance  of  stress 

♦  increase  or  decrease  timing 
interval  between  tasks 

Consistent  vs.  erratic  switching 

♦  Type  A  Behavior  Pattern 

♦  vary  the  pacing  of 
tasks/objects 

Task  duration 

♦  working  memory 

♦  some  task  should  be  quick; 
some  should  take  time 

Interruption  frequency 

♦  inhibition 

♦  working  memory 

♦  increase  or  decrease  number 
of  external  interruptions 

25 


As  outlined,  the  proposed  MTAT  assessment  tool  will  incorporate  two  of  the  major 
dimensions  featured  in  our  model  of  MT  environments:  type  of  MT  and  intensity  of  MT.  It  will 
not,  however,  address  the  third  dimension  noted  in  the  model,  severity  of  consequences.  It  is  not 
realistic,  in  a  brief,  abstract,  performance-based  test,  to  assess  how  severity  of  consequences  and 
risk-taking  preferences  affect  MT  performance.  There  would  be  no  way  to  realistically  create  an 
environment  where  failure  to  correctly  perform  a  task  would  have  consequences  severe  enough 
to  potentially  affect  MT  performance.  However,  this  is  not  a  major  shortcoming  of  the 
assessment  tool.  There  are  other  well-established  assessments,  such  as  personality  inventories, 
that  the  employer  likely  already  has  access  to  that  would  more  directly  assess  that  individual 
difference  variable  if  necessary. 

Assessment  Reports 

A  most  important  function  of  the  MTAS  would  be  to  provide  information  to  organizations 
about  job  candidates.  Therefore,  the  MTAS  also  includes  a  reporting  function  that  will  produce  a 
variety  of  useful  output  reports  that  describe  the  performance  of  an  individual  who  has  taken  a 
MTAS  test(s).  After  an  individual  completes  the  assessment  procedure,  the  test  administrator 
would  then  be  provided  with  a  profile  of  the  individual’s  MT  ability  and  how  it  relates  to  others 
who  have  taken  the  test.  An  individual’s  profile  package  would  include  the  individual’s  raw 
scores  for  each  type  of  test  (MTAT-DM,  MTAT-TFM,  MTAT-IM,  or  all  three)  at  each  level  of 
intensity.  It  would  also  provide  a  percentile  ranking  score  to  show  how  the  individual  compares 
to  others  who  have  taken  the  test.  Accompanying  the  individual’s  data  would  be  an  expectancy 
chart,  which  will  illustrate  the  relationship  between  a  particular  MTAT  score  and  actual 
workplace  performance.  Both  the  percentile  rank  and  the  expectancy  charts  will  be  created  using 
data  obtained  from  the  validation  and  reliability  studies. 

For  example,  an  individual  who  took  the  MTAT-DM  might  have  a  profile  that  looks  like 
the  following  chart. 


Test:  MTAT-DM 

Raw 

Score 

Percentile 

Rank 

High  Intensity  Level 

66 

85 

Moderate  Intensity  Level 

83 

90 

Low  Intensity  Level 

91 

95 

Table  7  shows  a  hypothetical  expectancy  chart  that  could  be  included  in  the  MTAS.  The 
hypothetical  expectancy  chart  shows  that  among  those  people  who  took  the  MTAT-DM  and  are 
employed  in  a  particular  job,  95%  of  the  people  who  scored  in  the  top  20%  on  the  MTAT-DM 
high  intensity  measure  were  rated  “above  average”  on  a  separate  performance  criterion  such  as 
supervisory  ratings  of  on-the-job  performance.  Given  that  the  MTAS  is  intended  to  be  applicable 
for  a  wide  variety  of  jobs,  it  would  not  be  possible  to  create  a  cut-off  score  for  the  selection  of 
candidates  for  each  potential  job.  However,  it  would  be  possible  for  test  administrators  in  a 
particular  field  to  use  the  expectancy  charts,  for  example,  to  establish  their  own  cut-off  scores,  if 
they  so  choose,  such  as  selecting  a  percentile  score  where  95%  of  those  who  score  at  this  level 
would  be  expected  to  qualify  for  the  job  after  experience  and  training. 
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Table  7. 

A  Hypothetical  Expectancy  Chart. 


Highest 
MTAT  20% 

percentile  Next  2Q% 

Middle  20% 

Next  20% 

0%  20%  40%  60%  80%  100% 

Percent  of  those  receiving  ratings  “above  average”  on 
supervisory  ratings  of  overall  on-the-job  performance 


Conclusions 

The  flexibility  of  the  MTAS  allows  it  to  be  used  both  as  a  selection  tool  and  as  a  placement 
tool  for  prospective  job  candidates.  A  job  administrator  who  is  looking  for  candidates  who  can 
function  well  and  handle  the  multi-tasking  demands  of  a  particular  workplace  environment,  for 
example,  would  first  complete  the  ENVAT  survey  in  order  to  determine  which  version  of  the 
MTAT  is  most  appropriate  to  administer  to  applicants.  After  administering  the  test,  they  would 
then  get  a  report  on  the  applicants’  scores  at  the  various  levels  of  MT  intensity,  as  well  as 
information  relating  the  scores  to  statistical  data  on  how  the  scores  are  associated  with  predicted 
performance  in  that  particular  environment.  This  information  might  then  be  used  as  one  of  the 
tools  to  assess  the  candidate’s  fit  for  the  position.  Likewise,  a  job  placement  counselor  might 
administer  all  three  versions  of  the  MTAT  to  a  job  seeker,  then  use  information  provided  in  the 
test  manual  to  provide  guidance  about  which  workplace  environments  the  job  seeker  would  be 
best  suited  for.  For  example,  if  someone  scored  poorly  on  the  high-intensity  scale  of  the  task 
flow  management  version  of  MTAT,  then  that  person  might  take  that  into  consideration  when 
applying  for  jobs  that  require  the  ability  to  manage  several  ongoing  tasks. 

The  goal  of  the  Phase  I  research  was  to  determine  whether  a  valid,  reliable,  and  practical 
assessment  tool  could  be  developed  that  predicts  MT  performance  in  a  wide  variety  of  MT 
environments.  As  the  measurement  approach  outlined  above  indicates,  the  MTAS,  with  its 
flexible  ability  to  provide  tailored  assessments  of  particular  MT  environments  and  to  deliver  tests 
that  tap  the  demands  required  by  those  environments,  will  capably  serve  that  function. 


TEST  PURPOSE,  SCOPE  AND  FRAMEWORK _ 

Construction  of  the  MTAS  will  be  guided  by  the  current  testing  standards  set  forth  by  the 
American  Educational  Research  Association  (AERA),  American  Psychological  Association 
(APA),  and  the  National  Council  on  Measurement  in  Education  (NCME)  (1999).  The  first  phase 
of  test  development  focuses  on  establishing  clear  definitions  of  the  proposed  test’s  purpose  and 
scope.  A  framework  for  the  test  is  developed  that  extends  the  purpose  of  the  test  to  describe  the 
construct  to  be  measured.  The  framework  delineates  aspects  of  the  construct  that  are  targeted  by 
the  test.  What  follows  documents  the  intended  purpose,  scope,  and  framework  for  the  MTAS. 
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Standards  (AERA,  APA,  &  NCME,  1999)  that  are  relevant  to  the  points  made  in  this  section  are 
given  in  parentheses. 

Purpose 

The  MTAS  will  serve  a  scientific  measurement  purpose  that  can  be  practically  used  to 
address  applied  needs  in  MT  environments.  Broadly  stated,  the  purpose  of  the  test  will  be  to 
measure  individual  differences,  within  normal  populations,  in  multi-tasking  ability,  tailored  to 
different  types  of  MT  environments  that  vary  on  key  dimensions.  The  MTAS  will  also  assess  the 
level  of  intensity  of  MT  that  an  individual  can  capably  handle.  In  so  doing,  the  test  can  be  used 
to  identify  those  individuals  who  are  likely  to  perform  well  in  different  kinds  of  environments  or 
jobs  that  require  varying  levels  of  MT  ability.  The  test  will  incorporate  a  scoring  system  that 
predicts  measures  of  asymptotic  performance  in  real-world  MT  environments,  as  well  as 
measures  of  time  required  to  reach  asymptotic  levels.  Hence,  it  will  be  both  a  test  of  ultimate 
performance  and  a  test  of  skill  acquisition.  (Standard  3.2) 

MT  ability  is  a  psychological  construct  that  has  received  increasing  attention  in  the  basic 
and  applied  literature  (e.g.,  Burgess,  2000;  Burgess,  Veitch,  de  Lacy  Costello,  &  Shallice,  2000; 
Joslyn  &  Hunt,  1998;  Meyer  &  Kieras,  1997;  Proctor,  Wang,  &  Pick,  1998;  Yee,  Hunt,  and 
Pellegrino,  1991)  (Standard  3.1).  Simply  stated,  the  MT  construct  is  the  ability  to  concurrently 
perform  or  interleave  multiple  tasks.  MT  ability  is  thought  to  place  heavy  demands  on  several 
executive  control  functions,  which  many  theoretical  accounts  include  as  part  of  working  memory 
(Burgess,  2000;  Burgess,  Veitch,  de  Lacy  Costello,  &  Shallice,  2000).  Despite  its  probable 
overlap  with  the  working  memory  construct,  current  findings  indicate  that  MT  ability  is  a  distinct 
individual  difference  variable  (Joslyn  &  Hunt,  1998).  Current  findings  also  indicate  that  it  has 
little  to  no  relationship  to  other  constructs  such  as  processing  speed  and  fluid  intelligence  (Joslyn 
&  Hunt,  1998).  These  conclusions,  however,  warrant  further  investigation  for  reasons  previously 
discussed.  MT  ability  also  incorporates  the  ability  to  prioritize  the  many  tasks  that  must  be 
performed.  A  body  of  research  exists  that  supports  the  existence  of  individual  differences  in  the 
ability  to  concurrently  perform  or  interleave  multiple  tasks.  Recent  research  (Joslyn  &  Hunt, 
1998)  has  succeeded  in  measuring  such  differences  and  predicting  performance  in  real-world 
environments  and  jobs  that  require  individuals  to  use  the  ability.  The  test  will  be  based  on  a 
recently  developed  laboratory  task  of  time-pressured  decision-making  (Joslyn  &  Hunt,  1998; 
Fischer  &  Mautone,  2005;  Morrin  &  Fischer,  2005)  that  has  been  shown  to  be  highly  predictive 
of  simulated  emergency  dispatching  and  ATC  job  performance.  (Standard  1.2,  3.2) 

Scope 

The  test  is  intended  to  discriminate  differences  in  MT  ability  among  normal  populations  of 
adults.  Although  a  body  of  research  has  associated  MT  ability  with  dysexecutive  syndrome  and  a 
variety  of  other  neuropsychological  disorders  that  involve  impairment  of  executive  control 
functions  (Burgess,  1998;  Burgess,  2000;  Burgess,  Veitch,  De  Lacy  Costello,  &  Shallice,  2000; 
Shallice  &  Burgess,  1991;  Wilson,  Evans,  Emslie,  Alderman,  &  Burgess,  1998),  the  test  is  not 
intended  as  an  instrument  to  diagnose  or  otherwise  measure  such  disabilities.  The  test  is  intended 
for  adult  populations  who  work  in  real-world  MT  environments,  and  should  not  be  used  to 
discriminate  differences  among  children  or  aged  populations.  The  test  is  also  intended  to  have 
limited  generalizability  with  respect  to  work  environments.  It  is  intended  to  predict  relevant 
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measures  of  performance  in  MT  environments,  but  not  in  stressful,  fast  paced,  nor  time-limited 
environments;  however  similar  these  environments  may  be  to  MT  jobs.  (Standard  1.2,  3.2) 

Framework 

The  present  research  provides  a  logical  framework  for  understanding  MT  ability  and  the 
proposed  MT  ability  test  (Standard  3.1).  Standards  recognize  that  this  framework  may  change  as 
test  development  proceeds  through  the  interplay  between  construct  development  and  test 
development  (AERA,  APA,  NCME,  1999).  However,  current  analysis  supports  basing  the  MT 
ability  test  on  the  cognitive  requirements  commonly  found  in  real-world  MT  jobs  and  the 
cognitive  requirements  that  different  MT  environments  place  on  workers.  Hence,  the  MT  ability 
test  will  incorporate  cognitive  operations  that  current  analysis  shows  are  critical  to  successful 
MT  performance.  The  cognitive  operations  that  appear  to  be  critical  are  short-term  memory 
rehearsal  and  storage,  working  memory  updating,  prospective  memory,  divided  attention, 
selective  attention,  mental  set  switching,  long-term  memory  retrieval,  and  prioritization. 

Analysis  of  the  MTAT  reveals  that  its  current  version  incorporates  and  requires 
participants  to  employ  a  set  of  cognitive  operations  that  are  a  good  match  to  the  operations 
required  by  MT  environments.  Short-term,  prospective  and  working  memory  operations  are 
integral  to  the  MTAT.  Executive  control  functions  such  as  mental  set  switching,  selective 
attention,  divided  attention,  and  rehearsal  for  STM  are  also  required  by  MTAT. 

The  ability  to  effectively  prioritize  multiple  tasks  appears  to  be  a  critical  function  that 
workers  must  perform  in  MT  environments.  While  the  ability  to  effectively  prioritize  multiple 
tasks  in  the  real  world  is  what  makes  or  breaks  a  worker,  we  currently  do  not  know  if  MTAT  can 
be  performed  relatively  successfully  without  this  skill.  However,  it  may  be  possible  to  increase 
the  degree  to  which  MTAT  measures  the  ability  to  prioritize  tasks  by  modifying  MTAT’s 
structure,  scoring  system,  or  rules.  The  importance  of  prioritization  to  real-world  performance  in 
MT  jobs  warrants  investigation  of  modifications  to  MTAT  to  better  represent  the  ability  to 
effectively  perform  this  operation. 


PHASE  II  TECHNICAL  OBJECTIVES  AND  APPROACH _ 

The  contributions  of  the  Phase  I  research  noted  above  lay  the  groundwork  for  further 
development  of  a  multi-tasking  assessment  system  that  assesses  various  MT  environments,  then 
provides  the  most  appropriate  test  to  predict  individuals’  aptitude  for  performing  well  in  a 
particular  MT  environment.  This  section  of  the  report  describes  the  technical  objectives  of  future 
research. 

The  proposed  Phase  II  research  is  designed  to  support  development  of  a  Multi-tasking 
Assessment  System  (MTAS).  The  concept  of  a  system  that  both  assesses  candidate  MT  work 
environments  and  provides  a  set  of  validated  tests  tailored  to  predict  performance  in  various  MT 
environments  is  truly  innovative.  Hence,  the  proposed  research  and  development  effort  is  not 
without  risk.  Successful  development  of  the  MTAS  depends  on  our  ability  to  create  a  reliable 
assessment  instrument  that  discriminates  MT  environments  on  key  variables.  It  also  depends  on 
the  ability  to  identify  those  key  environmental  variables  most  important  to  individual  differences 
that  determine  performance  in  different  MT  environments.  The  present  research  created 
assessment  and  model  products  that  meet  these  requirements.  However,  those  products  must  be 
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validated  to  ensure  that  the  MTAS  created  in  future  research  can  be  trusted  to  deliver  clear  and 
accurate  evaluations  of  environments  and  individuals. 

The  Phase  II  research  constitutes  a  highly  efficient  strategy  to  bring  the  MTAS  to  fruition. 
We  consider  its  objectives  to  be  the  necessary  groundwork  that  must  underlie  the  design  and 
development  of  the  MTAS.  The  research  is  designed  around  four  technical  objectives.  The  first 
technical  objective  of  the  project  is  to  perform  the  research  necessary  to  develop  a  reliable 
assessment  instrument  that  evaluates  candidate  MT  environments.  We  will  use  the  assessment 
instrument  developed  in  the  Phase  I  research  as  a  starting  point,  but  will  conduct  the  necessary 
development  and  investigation  to  refine  it  for  use  on  the  Internet.  The  second  technical  objective 
is  to  evaluate,  validate,  and  refine  the  model  of  MT  environments  developed  in  the  Phase  I 
research.  To  meet  this  objective,  we  will  conduct  a  wide  scale  evaluation  of  environments  using 
the  newly  developed  assessment  instrument  created  when  the  first  technical  objective  is  met.  The 
third  technical  objective  is  to  develop  the  structure  of  the  MTAS  and  all  testing  and  assessment 
components.  Realization  of  the  objective  will  be  a  prototype  MTAS  that  is  ready  for  validation. 
The  final  and  fourth  technical  objective  is  to  validate  the  MTAS  as  rigorously  as  possible  within 
the  constraints  of  a  Phase  II,  two-year  effort.  Three  validation  studies  are  proposed  that  target  the 
most  important  issues  concerning  the  validity  of  the  MTAS.  We  consider  the  proposed  research 
to  be  a  minimum  effort  to  make  the  MTAS  a  reality.  Because  the  proposed  research  both 
advances  the  science  of  MT  and  addresses  real  practical  concerns,  we  anticipate  that  basic  and 
applied  researchers  will  be  motivated  to  conduct  additional  research  once  our  work  is  published. 
Therefore,  it  is  likely  that  the  proposed  Phase  II  work  will  have  significant  effects  on  future 
knowledge  and  practice. 

The  work  plan  has  been  designed  to  achieve  the  first  three  technical  objective  in  the  first 
year  of  the  Phase  II  research.  The  fourth  technical  objective  will  be  met  by  conducting  multiple 
validation  studies  in  the  second  year  of  the  project. 


OVERVIEW  OF  PHASE  II  WORK  PLAN _ 

Development  of  a  validated  Multi-Tasking  Assessment  System  (MTAS)  is  the  main  driver 
to  the  proposed  work  plan.  Figure  5  depicts  the  progression  of  the  five  major  tasks  we  propose  to 
accomplish  in  the  project,  and  the  associated  deliverables.  The  plan  recognizes  and  addresses  the 
need  to  base  development  of  the  system  on  a  sound  body  of  empirical  research  and  tested  theory. 
Capitalizing  on  a  successful  Phase  I,  we  will  begin  by  testing  and  refining  the  assessment 
instrument  that  was  used  to  analyze  various  multi-tasking  environments. 

In  Task  1,  an  assessment  tool  will  be  developed  that  will  be  web-based  and  self- 
administering.  The  purpose  of  the  tool  will  be  to  assist  new  users  of  the  MTAS  in  defining  their 
particular  MT  environment.  Based  on  users’  responses,  the  tool  will  (1)  analyze  and  classify  the 
environment  along  key  dimensions,  (2)  provide  a  synopsis  of  the  environment  to  the  user,  and 
(3)  guide  the  users  to  the  appropriate  tests  for  their  environment,  which  may  be  administered  for 
selection  or  placement  purposes. 

In  Task  2,  research  will  be  performed  that  is  necessary  to  support  development  of  the 
MTAS.  The  purpose  of  this  research  is  twofold.  First,  it  will  serve  to  validate  critical  theoretical 
propositions  made  by  the  nascent  model  of  MT  environments  developed  in  the  Phase  I  research. 
Second,  the  research  will  serve  as  a  testing  ground  for  the  newly  developed  assessment  tool, 
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created  in  Task  1.  Based  on  the  studies’  findings,  the  model  of  MT  environments  and  the 
assessment  tool  will  be  revised  as  necessary.  These  refined  elements  will  then  be  used  to  adjust 
design  of  the  MTAS. 


Tested  &  Revised  ENVAT 


Tested  and  Revised 
Environment  Model 


Taskl:  Develop  ENVAT 

1.1  Refine  Current  Assessment  Instrument 

1 .2  Design  Usability  Study 

1 .3  Recruit  Sample 

1 .4  Conduct  Usability  Study 

1 .5  Analyze  Results;  Prepare  Report 

1.6  Revise  ENVAT 

- - 1 - 

Task  2:  Test  MT  Environment  Model 

2.1  Finalize  Design  of  Model  Validation  Study 

2.2  Recruit  Sample 

2.3  Conduct  Model  Testing  Study 

2.4  Analyze  Results;  Prepare  Report 

2.5  Revise  MT  Environment  Model 


Report  1 : 
Usability  Study 


Report  2: 

Model  Validation  Study 


Task  3:  Develop  MTAS 

|  3.1  Finalize  Design  of  MTAS 

Report  3: 

Prototype  MTAS 

r-  3.2  Develop  MTAS  Test  Components 

MTAS  Test  Manual 

\  3.3  Develop  MTAS  Structural  Components 

3.4  Develop  MTAS  Test  Manual 

Year  1 : 

1 

Interim  Report 

* — 

Task  4:  i 

i  -  -  -  —  -  ■  — -  -  — 

Conduct  Validation  Studies  of  MTAS 

Tested  &  Revised  MTAS 


4.1  Finalize  Designs  of  Validation  Studies 

4.2  Recruit  Samples 

4.3  Conduct  Studies 

4.4  Analyze  Results;  Prepare  Report 

4.5  Revise  MTAS 


Report  4: 

MTAS  Validation  Studies 


Packaged 
MTAS  Software 


Task  5:  Provide  Deliverables 

5.1  Final  Briefing 

5.2  Deliver  MTAS  Software 

5.3  Issue  Final  Report 


Report  5: 
Final  Report 


Figure  5.  Progression  of  tasks  to  complete  Phase  II  and  deliverables. 

The  Task  2  products  will  be  used  to  guide  development  of  a  set  of  tests,  to  be  created  in 
Task  3.  The  tests  constitute  key  components  of  the  MTAS  designed  to  evaluate  and  predict 
individual  performance  in  a  variety  of  MT  environments  that  vary  in  intensity  and  kind, 
according  to  the  model.  Structural  components  of  the  MTAS  will  also  be  developed  in  Task  3, 
such  as  user  interfaces,  algorithms  that  dictate  sequencing  of  tests,  and  linkages  between  user 
input  and  reports.  Finally,  a  test  manual  will  be  developed  in  Task  3  that  provides  requisite 
information  to  the  test  administrator  according  to  current  testing  standards  [American 
Educational  Research  Association  (AERA),  American  Psychological  Association  (APA),  and  the 
National  Council  on  Measurement  in  Education  (NCME),  1999]. 
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Once  the  tests  have  been  completed,  validation  studies  of  the  MTAS  will  be  conducted 
(Task  4).  The  validation  studies  will  serve  multiple  purposes.  For  example,  a  study  will  be 
conducted  to  determine  whether  the  MTAS  differentially  predicts  performance  in  different  types 
of  MT  environments,  as  designed.  The  validation  studies  will  also  address  the  following 
questions.  Does  it  make  sense  to  vary  the  intensity  of  the  testing  instruments  so  as  to  predict 
performance  in  low  and  high  intensity  MT  environments?  Does  it  make  sense  to  include 
component  tests  that  make  varying  cognitive  and  performance  demands  on  the  test  taker, 
analogous  to  environmental  variations?  Do  the  tests  predict  the  time  it  takes  to  learn  the  job, 
initial  performance,  or  ultimate  asymptotic  performance?  The  project  will  conclude  with  a  final 
briefing,  packaging  and  delivery  of  the  testing  software  and  documentation,  and  a  final  report 
(Task  5).  ' 


32 


REFERENCES 


About.com.  (2005).  Army  enlisted  job  descriptions  and  qualification  factors,  located  at: 
http://usmilitary.about.eom/od/enlistedjobs/a/  aijobs.htm 

Ackerman,  P.  (1992).  Predicting  individual  differences  in  complex  skill  acquisition:  Dynamics  of 
ability  determinants.  Journal  of  Applied  Psychology,  77,  598-614. 

American  Educational  Research  Association,  American  Psychological  Association,  &  National 
Council  on  Measurement  in  Education.  (1999).  Standards  for  educational  and 
psychological  testing. 

Bogner,  M.  S.  (Ed.).  (1994).  Human  error  in  medicine.  Hillsdale,  NJ:  Lawrence  Erlbaum 
Associates  Publishers. 

Brookings,  J.  B.,  &  Damos,  D.  L.  (1991).  Individual  differences  in  multiple-task  performance.  In 
D.  L.  Damos  (Ed.),  Multiple  task  performance  (pp.  363-386).  Washington,  DC:  Taylor  & 
Francis. 

Burgess,  P.W.  (1998).  Real-world  multitasking  from  a  cognitive  neuroscience  perspective.  In 
D.L.  Damos  (Ed.),  Control  of  cognitive  processes:  Attention  and  performance  xviii. 
Cambridge,  MA:  MIT  Press. 

Burgess,  P.  (2000).  Strategy  application  disorder:  The  role  of  the  frontal  lobes  in  human 
multitasking.  Psychological  Research,  63,  279-288. 

Burgess,  P.,  Veitch,  E.,  de  Lacy  Costello,  A,  &  Shallice,  T.  (2000).  The  cognitive  and 
neuroanatomical  correlates  of  multitasking.  Neuropsychologia,  38,  848-863. 

Cepeda,  N.  J.,  Kramer,  A.  F.  &  Gonzales  de  Sather,  J.  C.  M.(2001).  Changes  in  executive  control 
across  the  lifespan:  Examination  of  task-switching  performance.  Developmental 
Psychology,  37,  715-730. 

Chiles,  W.D.,  Jennings,  A.  E.,  &  West,  G.  (1972)  Multiple  task  performance  as  a  predictor  of  the 
potential  of  air  traffic  controller  trainees.  Office  of  Aviation  Medicine  Report  #FAA- 
AM-74-10. 

Comstock,  J.R.,  &  Amegard,  R.J.  (1992).  The  multi-attribute  task  battery  for  human  operator 
workload  and  strategic  behavior  research  (NASA  Tech.  Memo  No.  104174).  Hampton, 
VA:  National  Aeronautics  and  Space  Administration,  Langley  Research  Center. 

Conte,  J.  M.,  Rizzuto,  T.  E.,  &  Steiner,  D.  D.  (1999).  A  construct-oriented  analysis  of  individual- 
level  polychronicity.  Journal  of  Managerial  Psychology,  74(3/4),  269. 

Conte,  J.M.  &  Jacobs,  R.R.  (2003).  Validity  evidence  linking  polychronicity  and  Big  Five 
Personality  dimensions  to  absence,  lateness,  and  supervisory  performance  ratings.  Human 
Performance,  16,  107-129. 

Cook,  R.  L.,  &  Woods,  D.  D.  (1994).  Operating  at  the  sharp  end:  The  complexity  of  human 
error.  In  M.  S.  Bogner  (Ed.),  Human  error  in  medicine.  Hillside,  NJ:  Lawrence  Erlbaum 
Associates  Publishers. 

Damos,  D.  &  Smist,  T.  (1982).  Individual  differences  in  multi-task  response  strategies.  Aviation, 
Space,  and  Environmental  Medicine,  53,  1177-1181. 


33 


De  la  Casa,  L.  G.,  Gordillo,  J.  L.,  Mejias,  L.  J.,  Rengel  F.,  &  Romero  M.  F.  (1998).  Attentional 
strategies  in  Type-A  individuals.  Personality  &  Individual  Differences,  24,  59-69. 

Delbridge,  K.  (2001).  Individual  differences  in  multi-tasking  ability:  Exploring  a  nomological 
network.  Unpublished  Doctoral  Dissertation. 

Dickman,  S.  (1990).  Functional  and  dysfunctional  impulsivity:  Personality  and  cognitive 
correlates.  Personality  and  Individual  differences,  58,  95-102. 

Dickman,  S.  J.,  &  Meyer,  D.  E.  (1988).  Impulsivity  and  speed-accuracy  tradeoffs  in  information 
processing.  Journal  of  Personality  and  Social  Psychology,  54,  274-290. 

Fischer,  S.  C.,  Morrin,  K.  A.,  &  Joslyn,  S.  (2003).  Measuring  multi-tasking  ability  (No. 
ADA417039  (DTIC)).  Santa  Barbara,  CA:  Anacapa  Sciences,  Inc. 

Fischer,  S.C.  &  Mautone,  P.D.  (2005).  An  online  test  of  multi-tasking  ability.  Manuscript 
submitted  for  publication. 

Franklin,  A.  L.,  &  Hunt,  E.  (1993).  An  emergency  situation  simulator  for  examining  time- 
pressured  decision  making.  Behavior  Research  Methods,  Instruments,  &  Computers, 
25(2),  143-147. 

Gonzales,  C.  (2004).  Learning  to  make  decisions  in  dynamic  environments:  Effects  of  time 
constraints  and  cognitive  abilities.  Human  Factors,  46,  449-460. 

Gopher,  D.  (1982).  A  selective  attention  test  as  a  predictor  of  success  in  flight  training.  Human 
Factors,  24,  173-183. 

Heil,  M.C.  (1999).  Air  traffic  control  specialist  age  and  cognitive  test  performance.  Office  of 
Aviation  Medicine  Federal  Aviation  Administration  report.  DOT/FAA/AM-99/23. 

Ishizaka,  K.,  Marshall,  S.P.,  &  Conte,  J.M.  (2001).  Individual  differences  in  attentional  strategies 
in  multi-tasking  situations.  Human  Performance,  14,  339-358. 

Jemeic,  Z.  &  Sverko,  B.  (1994).  Time-sharing  factors  and  their  relation  to  cognitive  abilities  and 
personality  traits.  Personality  and  Individual  Differences,  16,  297-308. 

Joslyn,  S.,  &  Hunt,  E.  (1998).  Evaluating  individual  differences  in  response  to  time-pressure 
situations.  Journal  of  Experimental  Psychology:  Applied,  4(1),  16-43. 

Kahneman,  D.  (1973).  Attention  and  effort.  Englewood  Cliffs,  NJ:  Prentice-Hall. 

King,  R.E.,  Retzlaff,  P.D.,  Detwiler,  C.A.,  Schroeder,  D.J.,  &  Broach,  D.  (2003).  Uses  of 
personality  assessment  measures  in  the  selection  of  air  traffic  control  specialists.  Federal 
Aviation  Administration  report  DOT/FA  A/ AM-03/20. 

Lieberman,  M.  D.,  &  Rosenthal,  R.  (2001).  Why  introverts  can't  always  tell  who  likes  them: 
Multitasking  and  nonverbal  decoding.  Journal  of  Personality  and  Social  Psychology, 
80(2),  294-310. 

Meyer,  D.  E.,  &  Kieras,  D.  E.  (1997).  A  computational  theory  of  executive  cognitive  processes 
and  multiple-task  performance:  Part  1.  Basic  mechanisms.  Psychological  Review, 
104(1),  3-65. 


34 


Miyake,  A.,  Friedman,  N.  P.,  Emerson,  M.  J.,  Witzki,  A.  H.,  Howerter,  A.,  &  Wager,  T.  D. 
(2000).  The  unity  and  diversity  of  executive  functions  and  their  contributions  to  complex 
“frontal  lobe”  tasks:  A  latent  variable  analysis.  Cognitive  Psychology,  41,  49-100. 

Morrin,  K.A.,  &  Fischer,  S.C.  (2005).  Response  format  for  an  online  test  of  multi-tasking  ability. 
Manuscript  submitted  for  publication. 

Proctor,  R.  W.,  Wang,  D.  Y.,  &  Pick,  D.  F.  (1998).  An  empirical  evaluation  of  the  synworkl 
multiple-task  work  environment.  Behavior  Research  Methods,  Instruments,  & 
Computers,  30(2),  287-305. 

Rubinstein,  J.  S.,  Meyer,  D.  E.,  &  Evans,  J.  E.  (2001).  Is  multitasking  more  efficient?  Shifting 
mental  gears  costs  time,  especially  when  shifting  to  less  familiar  tasks.  Studying  the 
“Inner  CEO”  can  improve  interface  design,  personnel  training  and  diagnosis  of  brain 
damage.  American  Psychological  Association  Press  Release,  August  5,  2001. 

Schumacher,  E.  H.,  Seymour,  T.  L.,  Glass,  J.  M.,  Fencsik,  D.  E.,  Lauber,  E.  J.,  Kieras,  D.  E.,  & 
Meyer,  D.  E.  (2001).  Virtually  perfect  time  sharing  in  dual-task  performance:  Uncorking 
the  central  cognitive  bottleneck.  Psychological  Science,  12(2),  101-108. 

Shallice,  T.,  &  Burgess,  P.  (1991).  Deficits  in  strategy  application  following  frontal  lobe  damage 
in  man.  Brain,  114,  727-741. 

Sohn,  M.,  &  Anderson,  J.  (2001).  Task  preparation  and  task  repetition:  Two-component  model 
of  task  switching.  Journal  of  Experimental  Psychology:  General,  130,  764-778. 

Suss,  H.,  Oberauer,  K.,  Wittmann,  W.,  Wilhelm.  O.,  &  Schulze,  R.  (2002).  Working  memory 
capacity  explains  reasoning  ability  —and  a  little  bit  more.  Intelligence,  30,  261-288. 

Szymura,  B.  &  Necka,  E.  (1998).  Visual  selective  attention  and  personality:  An  experimental 
verification  of  three  models  of  extraversion.  Personality  and  Individual  Differences,  24, 
713-729. 

US  Army.  (2005).  Army  website,  Careers  &  Jobs,  located  at  http://www.goarmy.com/ 
JobCatList.do?fw=careerindex 

US  Department  of  Defense.  (1993).  America's  top  military  careers:  the  official  guide  to 
occupations  in  the  Armed  Forces.  Indianapolis,  IN:  JIST  Works. 

Wickens,  C.  D.  (1980).  The  structure  of  processing  resources.  In  R.  Nickerson  &  R.  Pew  (Eds.), 
Attention  and  performance  viii.  Hillsdale,  NJ:  Erlbaum. 

Wickens,  C.  D.  (1999).  Cognitive  factors  in  aviation.  In  F.  T.  Durso  (Ed.),  Handbook  of  applied 
cognition  (pp.  247-282).  Chichester,  England:  John  Wiley  &  Sons. 

Wilson,  B.A.,  Evans,  J.J.,  Emslie,  H.,  Alderman,  N.,  &  Burgess,  P.  (1998).  The  development  of 
an  ecologically  valid  test  for  assessing  patients  with  dysexecutive  syndrome. 
Neuropsychological  Rehabilitation,  8(3),  213-228. 

Yee,  P.L.,  Hunt,  E.,  &  Pellegrino,  J.W.  (1991).  Coordinating  cognitive  information:  Task  effects 
and  individual  differences  in  integrating  information  from  several  sources.  Cognitive 
Psychology,  23,  615-680. 


35 


APPENDIX  A:  INTERVIEW  INSTRUMENT 


Army  MT  Study 

SMEs  -  Phone  Interview  Questions 

1)  GENERAL  QUESTIONS: 

la)  Which  MOS’s  are  you  responsible  for? 

lb)  Just  so  we  are  on  the  same  page,  what  do  you  consider  to  be  multitasking? 

♦  need  to  emphasize  tasks  done  simultaneously  or  nearly  overlapping,  urgency, 
switching  back  and  forth 

lc)  On  a  scale  of  1-10  with  1  being  very  little  multitasking  and  10  being  a  high 
degree  of  MT,  how  would  you  rate  this  MOS? 


1 - 2 - 

• - 3- — 

-—4- — 

-—5- — 

- 6 - 

- - 7-—- 

-—8- — 

- 9 - 10 

low  degree 

high  degree 

2)  NUMBER  AND  TYPES  OF  TASKS 

2a)  What  specific  types  of  tasks  do  they  do? 

For  example:  a  Civilian  Police  Emergency  Dispatcher  must: 

♦  decide  which  police  units  to  send  to  an  incident  ♦  monitor  auditory  messages 

♦  contact  police  units  to  tell  them  where  to  go  ♦  look  up  information  in  a  database 

♦  monitor  incoming  messages  on  computer  screen  ♦  log  information 

♦  report  information  back  to  appropriate  police  unit  ♦  check  on  status  of  police  units 

♦  etc. 

□  resource  allocation  □  monitor  information  □  look  up  information 

□  log  information  □  give  directions  to  others  □  check  on  status 

□  interpret  data?  □  prepare  reports 
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2b)  Think  of  an  occasion  where  the  Soldier  is  engaging  in  a  high  level  of  MT.  How 
many  tasks  are  they  performing  at  any  given  period  of  time? 


2c)  Is  that  level  fairly  typical?  How  often  do  they  engage  in  MTing?  (frequently, 
occasionally,  rarely) 


2d)  Are  the  flow  of  tasks  fairly  steady  or  are  there  periods  of  low  MT  punctuated  by 
periods  of  intense  MT? 


2f)  Does  the  nature  of  the  tasks,  the  degree  of  MT  change  much  as  move  from  skill 
1,  to  skill  2  and  3? 


3)  SOURCES  OF  INCOMING  INFORMATION 

For  example:  Displays,  instruments,  communication  devices  such  as  radar  screens,  computer 
screens,  wind  velocity  readings,  visual  scanning  of  the  tarmac,  GPS,  radio  transmissions,  face-to- 
face  communication  with  other  people,  etc.  If  it’s  a  computer  screen  -  then  how  many  separate 
sources  of  information  does  it  provide  (i.e.,  is  it  split  into  more  than  one  window,  each  providing 
information  from  separate  sources?) 


3a)  For  this  MOS,  how  many  sources  of  does  the  Soldier  typically  have  to  interact 
with?  That  is,  pay  attention  to  and/or  check  on? 


3b)  Like  what? 
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3c)  How  many  do  they  typically  have  to  interact  with  at  the  same  time? 

Normally  - 

At  high  peaks  - 

3d)  Do  they  have  to  consciously  remember  to  check  the  information,  or  are  there 
cues  that  signal  when  to  check? 


4)  TASK  SWITCHING  MEMORY  (PROACTIVE,  RETROACTIVE,  ATTENTION) 


4a)  How  frequently  are  they  interrupted? 


1 - 2 - 3 - - 4 - 5 

Not  very  often  somewhat  often  Very  Often 

(1 5  or  so  minutes))  (every  few  minutes)  (every  few  seconds) 


4b)  How  frequently  does  a  Soldier  in  this  MOS  have  to  switch  to  another  task?  That 
is,  stop  what  they  are  doing  and  switch  to  another  task? 


1 - 2 - 3 - 4 - 5 

Not  very  often  somewhat  often  Very  Often 

(1 5  or  so  minutes)  (every  few  minutes)  (every  few  seconds) 

Relaxed  switching  Rapid  Switching 

Tasks  almost  done  in  serial  Tasks  done  almost  in  parallel 


4c)  Is  it  more  accurate  to  say  that  they  are  doing  several  tasks  at  one  time  or 
several  tasks  one  at  a  time? 


39 


4d)  For  this  MOS,  what  primarily  determines  when  a  Soldier  switches  to  a  new  task? 

For  example:  Sometimes  the  Soldier  has  no  choice  about  when  to  switch  tasks;  the 
environment  (incoming  call,  alarms,  specific  orders)  signal  when  to  switch.  Other  times  there 
are  no  specific  environmental  cues  that  and  it’s  entirely  up  to  the  Soldier  to  determine  when 
to  switch  tasks.  In  between  these  two  extremes  are  instances  where  there  are  some 
environmental  cues,  but  it’s  at  the  Soldier’s  discretion  about  whether  to  respond  to  or  ignore 
the  cues. 

So,  how  much  is  when  to  switch  tasks  dependent  upon  the  Soldier? 

1 - --2 - 3 - 4 - 5 

Not  at  all  Somewhat  Almost  Entirely 

(entirely  by  env.)  (Soldier  determines  when) 


4e)  In  some  jobs,  when  you  switch  back  and  forth  between  tasks,  you  have  to  be 
able  to  remember  what  you  were  doing  and  pick  up  where  you  left  off.  With 
other  jobs,  the  tasks  are  more  discrete  (for  example,  assigning  a  police  unit  to 
an  incident). 

Would  you  say  that  the  tasks  are  mostly  continuous  (that  is,  you  have  to 
remember  to  pick  up  where  you  left  off;  reorient  yourself  to  the  task)  or  more 
discrete  (when  you  resume  the  task,  you  don’t  really  need  to  remember  where 
you  left  off). 


On  a  scale  of  1-5,  how  continuous  is  the  task? 

1 - 2 - 3 - 4 - 5 

Not  continuous  (discrete)  Very  continuous 


4f)  How  dependent  are  the  tasks  on  one  another? 

For  example:  Do  you  need  to  take  what  you  are  doing  in  one  task  and  use  it  to  perform 
another  task?  For  example,  interpreting  radar  data,  then  using  that  information  to  give 
instructions  to  pilots  would  mean  that  the  data  interpretation  and  the  communication  tasks 
are  interdependent.  On  the  other  hand,  two  tasks  such  as  typing,  then  answering  the  phone 
would  be  independent  because  communicating  on  the  phone  does  not  depend  on  what  you 
were  typing.  They  are  two  separate  tasks. 


1 - 2 - 3- . 4 - 5 

Not  at  all  (Tasks  are  Tasks  are  somewhat  Tasks  are  very 

completely  interdependent  interdependent 

independent) 
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5)  PRIORITIZATION  OF  THE  TASKS  INVOLVED  IN  THIS  MOS: 

For  example:  If  a  job  has  task  prioritization,  it  means  that  there  are  primary  tasks  that 
require  more  attention  or  immediate  attention,  and  there  are  secondary  tasks  that  don’t 
require  as  much  attention  as  the  primary  task  -  or  can  be  postponed  in  order  to  perform  the 
primary  task.  If  a  job  has  little  or  no  prioritization,  then  most  tasks  are  equally  important  and, 
at  any  given  time,  and  if  two  or  more  tasks  need  to  be  done  it  usually  doesn’t  matter  which 
one  is  performed  first. 


5a)  Do  the  different  tasks  have  different  priorities?  That  is,  do  some  tasks  take 
priority  over  another? 


To  what  degree  is  there  task  prioritization? 

1 - 2 - 3 - 4 - 5 

not  much  some  a  great  deal 

(most  have  equal  priorities)  (very  different  priorities) 


5b)  Are  the  task  priorities  clearly  pre-defined? 

For  example:  Are  there  explicit  rules  and  procedures  about  which  task  takes  priority,  or  is  it 
sometimes  ambiguous  and  depends  on  the  situation?  For  example,  for  a  dispatcher,  sending 
out  units  is  top  priority,  logging  info  is  secondary;  for  a  manager,  sometimes  top  priority  is 


How  explicit  is  the  task  prioritization? 

1 - 2 - 3 - 4 - 5 

Not  very  Task  Priorities  Very 

(priorities  rarely  explicit)  somewhat  pre-defined  (clearly  pre-defined) 

(Follow-up:  ->  So,  if  someone  weren’t  very  skilled  at  prioritization,  what  impact  would 
that  have?) 


41 


5c)  How  stable  is  the  prioritization?  Is  it  fairly  stable  or  do  the  priorities  vary 
depending  on  the  situation? 

For  example:  If  prioritization  is  very  stable,  it  would  mean  that  one  task  is  always  first 
priority,  others  are  always  second  priority.  Somewhat  stable  would  mean  that  occasionally 
the  secondary  task  will  take  priority.  Rarely  stable  means  that  while  there  are  clearly  some 
tasks  that  need  to  take  precedence  over  others,  determining  which  task  is  most  important  at 
any  given  time  requires  constant  re-evaluation  and  assessment. 


1 - 2 - 3 . . 4 - 5 

Task  Priorities  Task  Priorities  Task  Priorities 

Very  stable  '  .  somewhat  stable  rarely  stable 


5d)  Who  primarily  decides  what  the  prioritization  is?  (Soldier,  Supervisor, 
Procedures) 


6)  LEVEL  OF  DECISION  MAKING 

For  most  of  the  tasks,  what  level  of  decision-making  is  required? 

For  example:  For  routine  or  automatic  decision-making,  there  are  probably  well-established 
rules  and  procedures  about  what  to  when  faced  with  a  particular  situation.  When  X  happens, 
do  Y.  On  the  other  hand,  when  deliberate  or  conscious  decision-making  is  required,  there 
may  not  be  prescribed  responses,  or  there  may  be  several  options  about  what  to  do  and  a 
decision  has  to  be  made  about  which  is  most  appropriate. 


6a)  How  routine  are  the  decisions? 

1 - 2 - 3 - 4 - 5 

Not  very  routine  Very  routine 

(conscious/deliberate)  (almost  automatic) 


6b)  Along  the  same  lines,  how  much  autonomy  does  the  person  have  in  making 
decisions? 

For  example:  Do  most  decisions  involve  following  established  procedures,  or  do  they 
involve  coming  up  with  potentially  novel  solutions  or  selecting  among  multiple  solutions? 

1 - 2 - 3 - 4 - 5 

Not  much  autonomy  A  great  deal  of  autonomy 

(mostly  prescribed  responses)  (Novel/multiple  potential  solutions) 
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6c)  How  rapidly  do  they  have  to  make  decisions?  Do  they  usually  have  to  decide 
quickly  or  do  they  have  time  to  consider  and  decide? 


1 - 2 

Not  very  rapidly 
(take  time  to  deliberate) 


- 3 - 

Somewhat  rapidly 


4 - 5 

Very  rapidly 
(No  time  to  deliberate) 


7)  DEGREE  OF  TASK  URGENCY 


7a)  Do  the  tasks  need  to  be  started  immediately,  or  could  they  be  temporarily  put 
off? 

How  urgent  are  most  of  the  tasks? 


1 - 2 

Not  very  urgent 
(can  wait  a  few  hours) 


- 3 - 

Somewhat  urgent 
(can  wait  a  few  minutes) 


4 - 5 

Very  urgent 

(need  immediate  attention) 


7b)  How  rushed  is  the  task?  How  quickly  do  they  have  to  work/react? 


1 - 

Not  rushed 
Can  take  their  time 


Very  Rushed 
Must  work  very  quickly 


7c)  What  is  the  typical  speed  vs.  accuracy  tradeoff? 

For  example:  Is  the  main  focus  on  getting  things  done  quickly  or  reacting  quickly  to  stimuli? 
Or  is  it  on  accuracy  and  taking  time?  Is  it  better  to  be  fast  and  sometimes  wrong  (e.g.,  have 
some  false  alarms)  or  is  it  better  to  take  more  time  and  be  accurate? 


How  important  is  speed? 

2 - 3 - 4 


1 - 

Very 


- 5 

Not  very 


7d)  What  makes  the  task  urgent? 

For  example: 

□  Time  schedule  that  needs  to  be  adhered  to 

□  People  depend  on  it 

□  May  forget  about  it 

□  Need  to  act  now  or  opportunity  is  lost 

□  Need  to  act  now,  or  there  could  be  serious  consequences 


8)  STRESS 

8a)  What  are  the  consequences  of  failure  to  successfully  perform  the  task? 

For  example:  loss  of  life,  money,  time,  inexpensive  material  resources,  expensive  material 
resources.  Danger  to  self  or  others,  personal  or  professional  embarrassment;  no  loss  (if  no 
one  finds  out),  no  loss  (even  if  someone  finds  out),  etc.  Someone  else  can  do  it,  you  can 
start  over. 


Please  List: 


9)  GENERAL  QUESTION  -  PART  II: 

9a)  Besides  their  ASVAB  scores,  what  characteristics  does  a  person  need  to  have 
in  order  to  do  well  in  that  MOS? 

(ex:  focused,  problem-solving,  openness  to  new  experiences,  situational  awareness,  not 
easily  distracted,  quick,  organized,  good  memory,  need  for  achievement  etc) 


Follow-ups: 

9b)  What  distinguishes  a  top  performer  from  a  mediocre  performer?  (Besides 
ASVAB  and  attitude)? 
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9c)  Can  you  think  of  a  case  where  someone  seemed  like  he  or  she  should  do  well, 
but  had  trouble  performing  the  MOS?  Why? 


9d)  Do  you  think  there  is  something  that  distinguishes  this  MT  environment  from 
other  MT  environments?  (That  someone  who  does  well  in  another  MT 
environment  would  also  do  well  here?) 


9e)  Are  there  other  jobs  that  are  very  similar  to  this  one,  but  that  you  would  consider 
to  require  either  more  or  less  MT  ability? 


Is  there  any  thing  else  you  would  like  to  add? 

Thank  you  very  much  for  helping  out  with  this  project.  If  you  have  any  additional 
questions  or  comments  please  let  me  know.  My  phone  number  is  805-966-6157  ext.  13. 

Could  I  have  your  e-mail  address  in  case  I  have  any  brief  follow-up  questions? 

I 
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APPENDIX  B:  SUMMARY  OF  INTERVIEW  RESPONSES 
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APPENDIX  C:  DETAILED  DESCRIPTIONS  OF  THREE  TYPES  OF  MT 
ENVIRONMENTS  DESCRIBED  BY  MODEL 


MT  Environment  Type  #1:  “Decision  Making” 


General  environment:  Provides  mostly  ill-defined  problems  and  changeable,  multiple  tasks 
that  require  a  variety  of  skills  and  an  extensive  specialized  knowledge  base. 

Typical  tasks:  seeking  and  gathering  information,  resource  allocation  decisions,  and 
making  decisions  about  which  actions  to  take 

General  Description  of  Requirements:  must  efficiently  and  purposefully  gather 
information  to  make  the  best  decisions  about  what  actions  to  take  and  how  to  allocate 
limited  resources  often  on  moment-to-moment  basis;  problems  require  complex  decision 
making 

Typical  Military  Jobs:  13F  Fire  Support  Specialist;  some  Health  Care  Specialists  (91W), 
Company  commander. 

Typical  Civilian  Jobs:  Some  nursing  positions  (ER),  ER  physician,  Chief  fire  fighter,  some 
police  positions. 

Decision-making:  Complex,  ill-defined  problems  that  have  multiple  satisfying  solutions. 
Decisions  must  be  quick  and  some  may  be  automated. 

Prioritization:  worker  is  responsible  for  prioritizing  many  urgent  tasks 

Autonomy  in  Task-switching:  High  autonomy  in  switching  among  tasks 

Pacing  and  Intensity:  can  be  fast  paced  and  intense,  or  may  be  moderately  paced  and  less 
intense;  probably  variable  pacing  within  each  job 

Consequences  of  failure:  varies;  might  be  low,  medium,  or  high 

Heavy  cognitive  demands  on:  fluid  intelligence,  ability  to  quickly  accumulate  and  integrate 
new  knowledge 

Essential  qualities  of  good  worker:  ability  to  prioritize,  make  quick/appropriate  decisions; 
to  learn  a  wide  variety  of  new  knowledge,  equipment,  and  procedures;  tolerate  stress 
associated  with  severe  consequences 

MT  Environment  Type  #2:  “Task  Flow  Monitoring” _ 


General  environment:  Fairly  well-defined  (except  when  something  unusual  happens  or 
when  somebody  makes  a  mistake);  multiple  concurrent  tasks  that  require  the  same  kind  of 
skills,  limited  number  of  sources  of  incoming  information. 

Typical  tasks:  after  initiation  of  tasks,  monitoring  progress,  checking  status,  executing 
prescribed  actions 

General  Description  of  Requirements:  keeping  many  continuous  activities  going  at  once 
Typical  Military  Jobs:  92G:  Food  Service  Operations,  some  systems  operator  positions 
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Typical  Civilian  Jobs:  Dining  Services  (dorms,  cafeterias),  some  nursing  positions  (floor 
nurse  in  certain  hospital  departments),  factory  supervisor,  emergency  dispatcher 

Decision  making:  fairly  routine;  not  a  heavy  emphasis  on  complex  decision  making 

Prioritization:  typically  prescribed 

Autonomy  in  Task-switching:  mainly  cued  by  environment,  schedule,  supervisor 

Pacing  and  Intensity:  can  be  fast  paced  and  intense,  or  may  be  moderately  paced  and  less 
intense,  but  mostly  steady. 

Consequences  of  failure:  varies;  might  be  low,  medium,  or  high 

Heavy  cognitive  demands  on:  prospective  memory,  organization  skills, 

Essential  qualities  of  good  worker:  time-management  skills,  ability  to  coordinate  tasks, 
ability  to  establish  and  follow  a  routine. 

MT  Environment  Type  #3:  “Monitor  Multiple  Sources  of  Information” _ 


General  environment:  Multiple  sources  of  information  that  must  be  integrated. 

Typical  tasks:  Listening  to  audio  messages,  monitoring  visual  displays,  interpreting 
information  to  make  assessments  or  to  extract  meaning,  possibly  integrating  sources  to 
deduce  or  derive  trends  or  classifications 

General  Description  of  Requirements:  Monitors  information  from  many  sources  and 
either  integrates  or  responds  to  each  bit  of  information.  There  is  less  of  an  emphasis  on 
actions  in  this  environment  and  more  of  an  emphasis  on  assessment,  integration,  and 
interpretation  of  multiple  information  sources. 

Typical  Military  Jobs:  98J:  Electronic  Intelligence  Interceptor/Analyst;  88N: 
Transportation  Management  Coordinator 

Typical  Civilian  Jobs:  Control  room  operator  in  power  plant,  pilot  under  certain  task 
conditions,  ICU  nurse 

Decision-making:  Fairly  and  typically  routine;  not  a  heavy  emphasis  on  complex  decision 
making,  except  when  something  goes  wrong. 

Prioritization:  Does  not  require  prioritization  to  the  same  degree  as  other  environments. 

Autonomy  in  Task-switching:  Mostly  directed  by  environmental  cues  (incoming 
information) 

Pacing  and  Intensity:  can  be  fast  paced  and  intense,  or  may  be  moderately  paced  and  less 
intense;  pacing  could  be  either  fairly  steady  or  erratic 

Consequences  of  failure:  varies;  might  be  low.  medium,  or  high 

Heavy  demands  on:  divided  or  focused  attention,  interpretation  and  integration  of 
information 

Essential  qualities  of  good  worker:  good  short-term  memory,  ability  to  allocate  attention 
effectively  -  and  adapt  to  the  requirements  of  the  situation. 
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